(19) 



EuropaischesH'atentamt 
European Patent Office 
Office' europeen des brevets 




(11) 



EP1 130 927 A2 



(12) 



EUROPEAN PATENT APPLICATION 





Date of publication: 


(51) lntCl7: H04N 7/50 




05.09.2001 Bulletin 2001/36 


/o^ \ 
(21) 


Application number: 01301850.2 




(22) 


uate oT Tiling: 28.02.2001 




(84) 


Designated Contracting States: 


• Ozaki.Seljl 




AT BE CH CY DE DK ES Fl FR GB QR IE IT LI LU 


Shinjuku-ku, Tokyo (JP) 




MC NLPT SE TR 


• Kasai, HIroyuki, c/o Global Inf. and Telec. 




Designated Extension States: 


Shinjuku-ku, Tokyo (JP) 




AL LT LV MK RO SI 


• Nishlmura, Satoshi, c/o Global Inf. and Telec. 


(30) 




Shinjuku-ku, Tokyo (JP) 


Priority: 02.03.2000 JP 2000057815 


• Tominaga, Hideyoshi, c/o Global inf. and Telec. 


(71) 




Shinjuku-ku, Tokyo (JP) 


Applicants: 


• 


IVIedia Glue Corporation 


(74) Representative: Cooper, John et al 




Tolcyo (JP) 


IVIurgitroyd & Company, 


• 


Tominaga, Hideyoshi 


Chartered Patent Agents, 




Tokyo (JP) 


373 Scotland Street 


(72) 




Glasgow G5 8QA (GB) 


Inventors: 


• 


Hanamura,Tsuyoshl 






Shinjuku-lcu, Tokyo (JP) 





(54) Apparatus, method and computer program product for transcoding a coded multiplex sound 
and moving pictuture sequence 



(57) Herein disclosed Is an apparatus 
(600,700,200) for transcoding a coded nnulttplexed 
sound and moving picture sequence comprising a rate 
converter, which Is operated to compress only the video 
bit streams, which have extremely large information vol- 
ume. The compression operation is carried out through 
the steps of (a) operating the MPEG-2 transport stream 
(TS) demultiplexer (61 0) to demultiplex the input MPEG- 
2 transport streams into video TS, audio TS, and system 
Infomnatlon TS; (b) operating the MPEG-2 video trans- 
coder (640) to transcode the video TS to generate out- 



put video TS having Information volume less than the 
video TS; (c) operating the system controller (650) to 
modify or not modify part of fixed codes of the system 
Information TS in accordance with the output MPEG-2 
transport streams; (d) operating the MPEG-2 TS multi- 
plexer (620) to multiplex the video TS thus transcoded, 
the audio TS, and the eventually partly-modified or not 
modified system Information TS into output MPEG-2 
transport streams, and output the output MPEG-2 trans- 
port streams, thereby making It possible to control the 
output bit rate less than the input bit rate. 
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Description 

BACKGROUND OF THE iNVENTION 

5 1 . Field of the Invention 

[0001] The present Invention relates to an apparatus, a method and a computer program product for transcoding a 
coded multiplexed sound and moving picture sequence, and more particularly, to an apparatus, a method and a com- 
puter program product for demultiplexing the first coded multiplexed sound and moving picture sequence signal con- 

10 talning video infonnation, audio infomriation, and program infomnation into video infomnation data strings, audio infor- 
mation data strings, and control infomnation data strings, compress the video information data strings, and multiplex 
the compressed video information data strings, the audio infonnalion data strings, and the control information data 
strings in to the second coded multiplexed sound and moving picture sequence signal, and thus converting a first coded 
multiplexed sound and moving picture sequence signal containing video information, the audio information, and the 

IS program infonnation in the form of bit streams transmitted at a first bit rate Into a second coded multiplexed sound and 
moving picture sequence signal in the torn of bit streams transmitted at a second bit rate lower than the first bit rate. 

2. Description of the Related Art 

20 [0002] There have so far been proposed a wide variety of systems for compressing and encoding a moving picture 
having a considerable large amount of data to produce a coded moving picture sequence signal. The international 
standard, ISO-IEC 1 381 8-2, was created for a system operable to encode a digital video signal associated with a digital 
audio signal and commonly called "Moving Picture Expert Group Phase 2", I.e., "MPEG-2". The MPEG-2 system is 
designed to encode a digital video signal associated with a digital audio signal to generate a coded multiplexed sound 

25 and moving picture sequence signal in the fomn of bit streams. For avoiding tedious repetition in the following descrip- 
tion, the bit streams confonnable to the above MPEG-2 standard will be referred to as "MPEG-2 bit streams", and a 
device for encoding a digital video signal associated with a digital audio signal to generate a coded multiplexed sound 
and moving picture sequence signal In the form of bit streams will be referred to as an "encoder", herelnlater In recent 
years, the MPEG-2 standard has become Increasingly applied for various technical systems such as a communicating 

30 system, and a television broadcasting system. 

[0003] The above MPEG-2 bit streams are each of a hierarchical structure consisting of: a top, sequence layer; a 
GROUP OF PICTURES layer; a picture layer; a slice layer; a macroblock layer; and a low, block layer. 
[0004] The typical encoder is operable under the MPEG-2 standard through a method of compressing and encoding 
a moving picture as follows. The method comprises the steps of: 

35 

(a) inputting the moving picture sequence consisting of a series of pictures; 

(b) temporally storing the series of pictures as frames in memories, respectively; 

(c) computing a difference between one frame and another frame to eliminate redundancy in a time axis direction; 
and 

-^0 (d) orthogonal transfomning, e.g., discrete cosine transforming (OCT), a plurality of picture elements within each 

of the frames to eliminate redundancy in a spatial axis direction. 

[0005] The encoder thus constructed can compress and encode the moving picture to generate and output a coded 
moving picture sequence signal in the fomi of the MPEG-2 bit streams through a transmitting path at a predetermined 
^5 bit rate. The coded signal is then transmitted from the encoder to a decoder which is adapted to decode the coded 
signal to reproduce the moving pfcture. The typical decoder Is operated to decode the coded signal through a so-called 
bi-directionally predicting method which comprises the steps of: 

(a) storing one reproduced picture, generally referred to as "intra-picture", i.e., "l-picture", In a first frame memory; 
50 (b) estimating another picture generally referredto as "predictive-picture", i.e., "P-picture", followed by the l-picture,' 

on the basis of the information on a difference between l-ptcture and P-picture; 

(c) storing the estimated picture In a second frame memory; and 

(d) estimating further another picture interposed between the l-picture and P-picture, generally referred to as "bi- 
directionally predictive-picture", i.e., "B-picture". 

55 

[0006] Here, the l-picture Is encoded independently of the pictures of the other types, so that a single l-plcture can 
be reproduced as a static Image only by itself. A P-picture can be predicted on the basis of the l-picture or a P-picture 
located on a position prior to the P-plcture to be encoded. 
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[0007] In the above encoder, a volume of Infonnatlon on the coded moving picture sequence signal is, however, 
variable. In particularly, the volume of infonnation increases remarkably when a scene Is changed. The decoder is 
generally provided with an input buffer for receiving the coded moving picture sequence signal from the encoder. The 
input buffer of the decoder, however, has a limited storage capacity. Therefore, when a large number of bits of the 

5 coded moving picture sequence signal are transmitted from the encoder to the decoder, the input buffer overflows with 
the bits of the coded moving picture sequence signal thereby making the decoder difficult to process the coded moving 
picture sequence signal. In order to transmit such coded moving picture sequence signal having a variable number of 
bits through the transmitting path at a predetermined bit rate and to make it possible for any decoder to receive the 
whole of the coded moving picture sequence signal without overflow, the encoder comprises: an output buffer for 

10 temporally storing the coded moving picture sequence signal before transmitting the coded moving picture sequence 
signal through the transmitting path; and a rate controller for controlling the volume of infonnation on the coded moving 
picture sequence signal stored in the output buffer so as to keep the volume of information on the coded moving picture 
sequence signal to be transmitted to the decoder from exceeding the volume of the input buffer of the decoder and 
then to control the bit rate of the coded moving picture sequence signal. 

15 [0008] A typical rate controlling method in MPEG-2 standard is described in "ISO-IEC/JTC1/SC29/WG1 1/N0400 Test 
Model 5", April, 1 993, hereinlater refen^ed to as TM-S". The rate controlling method according to the TM-5 comprises 
the steps of: 

(I) allocating a target number of bits to a picture of each type on the basis of the total number of bits available to 
20 the current pictures to be encoded in the GROUP OF PICTURES, i.e., R, which will be described hereinafter; 

(II) computing the reference value of a quantization parameter used for the quantization of each of macroblocks 
in a current picture on the basis of the utilization volume of a "virtual buffer" to perform the rate control; and 

(III) modulating the reference value of the quantization parameter in accordance with the spatial activity In the 
macroblock. 

25 

[0009] Furthermore, there are many types of decoders. For instance, a decoder is designed to decode the coded 
signal in a unique compression fomnat different from that of MPEG-2 bit streams, and another decoder is connectable 
to a transmitting path having a different bit rate. The decoders of these types are therefore required to provide with an 
apparatus, a so-called transcoder, for converting the MPEG-2 bit streams into another appropriate coded signal in the 
30 specified format having the required bit rate. The transcoder makes It possible for the encoder to transmit the coded 
signal to any types of decoders. 

[0010] Referring to Fig. 18 of the drawings, there Is shown a conventional transcoder of one typical type as a first 
conventional transcoder 50. The conventional transcoder 50 has an input terminal electrically connected to a first 
transmitting path, not shown, and an output temriinal electrically connected to a second transmitting path, not shown. 

35 The conventional transcoder 50 is designed to input first bit streams at a predetermined input bit rate through the 
input terminal , to convert the first bit streams b-, into second bit streams to be outputted at a predetermined output 
bit rate, i.e., a target bit rate, lower than the input bit rate of the inputted first bit streams b^, and then to output the 
second bit streams b2 through the output terminal The conventional transcoder 50 comprises a variable length 
decoder 51 , referred to as "VLD" in the drawings, an inverse quantizer 53, referred to as "10" in the drawings, a quantizer 

40 55, refen^ed to as "Q" in the drawings, a variable length encoder 57, referred to as "VLC" in the drawings, and a rate 
controller 59. 

[0011] The variable length decoder 51 is electrically connected to the input tennlnal and designed to decode a 
coded moving picture sequence signal within the first bit streams b^ Inputted through the Input terminal a^ to reconstruct 
original picture data for each of pictures including a matrix of original quantization coefficients, referred to as "level", 
45 for each of macroblocks within each of the pictures and an original quantization parameter, hereinlater referred to as 
"first quantization parameter Q/. 

[001 2] The inverse quantizer 53 Is electrically connected to the variable length decoder 51 and designed to input the 
matrix of original quantization coefficients level from the variable length decoder 51 and the first quantization parameter 
Q^. The inverse quantizer 53 is further designed to inversely quantize the inputted matrix of original quantization co- 
50 efficients level with the first quantization parameter to generate a matrix of de-quantization coefficients, referred to 
as "dequant", i.e., DCT coefficients, for each of macroblocks as follows: 



Q.xQM 

dequanH2x levels sign{tevef)]x-~ — equation (1) 



or 
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Q.xQM 

clequant=levelx -L^ — equation (2) 



5 where the equation (1 ) is used for the inter macroblock, while the equation (2) is used for the intra macroblock. 

QM is a matrix of quantization parameters stored in a predetermined quantization table. Thef irst quantization parameter 
and the matrix of quantization parameters QM are derived from the Inputted first bit streams b-, by the decoder 51 . 
Here, the original quantization coefficients level, the de-quantization coefficients dequant, the matrix of quantization 
parameters QM, and the first quantization parameter Q^ are integers. The de-quantization coefficients dequant cal- 

10 culated by the equations (1 ) and (2) should be rounded down to the nearest one. 

[001 3] The quantizer 55 is electrically connected to the inverse quantizer 53 and designed to input the matrix of de- 
quantization coefficients dequant from the inverse quantizer 53 and then q uantize the Inputted matrix of de-quantization 
coefficients dequant for each of macroblocks with a second quantization parameter, referred to as "Q2" herelnlater, 
to generate a matrix of re-quantization coefficients, referred to as "tlevel", as follows: 



tievehdequantx ^ equation (3) 



20 or 



16 1 
tievehdequantx -^-^^^+sign(dequant)x- equation (4) 

25 

where the equation (3) Is used for the inter macroblock, while the equation (4) is used for the intra macroblock. 
The second quantization parameter Qj is obtained by the rate controller 59. Here, the re-quantizatlon coefftelents tievel 
and the second quantization parameter Q2 are also integers. The re-quantization coefficients tievel calculated by the 
equations (3) and (4) should be rounded down to the nearest one. Such rounding operation for the integers will be 

30 omitted from the later description for avoiding tedious repetition. 

[0014] The variable length encoder 57 Is electrically connected to the quantizer 55 and designed to input the re- 
quantization coefficients tievel from the quantizer 55 and then encode the Inputted matrix of the re-quantlzatlon coef- 
ficients tievel to generate objective picture data for each of pictures to sequentially output the objective picture data 
in the form of the second bit streams b2 through the output terminal 82- The variable length encoder 57 Is further 

35 electrically connected to the variable length decoder 51 and designed to input a diversity of infomnation data included 
in the first bit streams b^ necessary for the second bit streams bj from the variable length decoder 51 . 
[001 5] The rate controller 59 is electrically connected to the inverse quantizer 53 and designed to perform rate control 
process In accordance with the TM-5 on the basis of the information obtained from the Inverse q uantlzer 53 as described 
below. 

40 [001 6] Referring to Fig. 1 9 of the drawings, there is shown a flowchart of the rate controlling process in accordance 
with the TM-5 carried out In the conventional transcoder 50. As shown In Fig. 1 9, the rate controlling process comprises 
steps A1 to A14. 

[001 7] In the step A1 , "1 " is assigned to a picture number variable n representing the serial number of a picture within 
the first bit streams b-,. Hereinlater, a n-th picture in the first bit streams is referred to as "plc(n)". 
45 [001 8] In the following step A2, a global complexity measure, referred to as X|, Xp, or X5, for a picture of the corre- 
sponding type, i.e., I, P or B-plcture is computed as follows: 



50 



X,=S, X Q, equation (5) 



or 



Xp=Sp X Qp equation (6) 

55 



or 
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Xb=Sb X equation (7) 

where S,, Sp, or is the number of bits generated for an encoded I, P or B-picture, and Q|, Qp, or Qj, is the 
^ average quantization parameter computed by averaging the actual quantization values used during the quantization 
of the all macroblocks within I, P or B-picture. The average quantization parameters Q|, Qp, and are normalized 
within a range of 1 to 31. The average quantization parameters Q,, Qp, and respectively correspond to the first 
quantization parameters Q^ obtained from the variable length decoder 51 . 

[0019] The global complexity measure X|, Xp, or of the corresponding picture is inversely proportional to the 
*o compressing ratio of the moving picture, namely, a ratio of the volume of infomriatlon in the second bit streams b2 to 
that in the first bit streams b^. Namely, as the volume of infomnation on the first bit streams b-, becomes larger, the 
compressing ratio is decreased. Therefore, the global complexity measure X|, Xp, or X^ of the corresponding picture 
becomes larger, as the compressing ratio Is decreased. In contrast, the global complexity measure X,, Xp, or X|, of the 
corresponding picture becomes smaller, as the compressing ratio is increased. 

[0020] The initial value of global complexity measure X,, Xp, or X^ of the corresponding picture is given as follows: 

X,=1 60xTarget_Bitrate/1 15 equation (8) 



IS 



20 



25 



30 



35 



40 



45 



or 

Xp=60 X Target_Bitrate/1 1 5 equation (9) 

or 

Xb=42 xTarget_Bitrate/1 1 5 equation (1 0) 

where Target^Bltrate Is measured In bits/s and corresponds to the target bit rate of the first conventional trans- 
coder 50. 

[0021] In the following step A3, the target number of bits for a picture of the corresponding type, i.e., I, P or B-picture 
to be encoded In the cun-ent GROUP OF PICTURES, referred to asT;, Tp, orT^ Is computed as: 

^^~NJCnJ<^ equation (11) 



^P" WbV^ equation (1 2) 



or 



H 

M +_P_b__P 



^b" M K y equation (13) 



50 b f^^x^ 

where Np and Nj, are the number of P-pictures and B-pictures remained not yet encoded in the current GROUP 
OF PICTURES, respectively Kp and are constants computed on the basis of the ratio of the quantization value of 
P-picture to the quantization value of l-picture, and the ratio of the quantization parameter of B-picture to the quantization 
value of l-picture, respectively. When it is assumed that Kp=1.0 and Kb=1.4, the quality of the image can be always 
optimized. 

[0022] In the following step A4, it is judged upon whether the picture number variable n is "1 " or not, i.e., the current 
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picture is the first picture plc(1) or not. When It is judged that the picture number variable n is "1", i.e., the current 
picture Is the first picture pic(1), the step A4 goes fonward to the step A5. When, on the other hand, it is judged that 
the picture number variable n is not "1", i.e., the current picture is not the first picture, the step A4 goes fonward to the 
step A6. In the step A5, the total number of bits available to the pictures to be encoded in the current GROUP OF 
PICTURES, i.e., the remaining number of bits available to the GROUP OF PICTURES, hereinlater referred to as R, is 
initialized in accordance with the following equation (14). This remaining number of bits available to the GROUP OF 
PICTURES R is computed before encoding the first picture plc(1) within the GROUP OF PICTURES, as follows: 

R=TargeLBitratexNPICpicture_rate-hR equation (14) 



where NPIC is the total number of pictures of any type in the GROUP OF PICTURES, and picture.rate Is ex- 
pressed in the number of pictures decoded and indicated per second. At the start of the sequence R=0. 
[0023] In the step A6, the above remaining number of bits available to the GROUP OF PICTURES R is updated 
IS before encoding the current picture pic(n) as follows: 

R=R-S, equation (15) 



or 



or 



R=R-Sp equation (16) 



=R-Sb equation (17) 



where S|, Sp, or is the number of bits generated in the previously encoded picture plc(n-1 ) of the con^esponding 
type (I, P or B). 

[0024] The step A5 or A6 goes fonA/ard to the step A7 wherein "1" is assigned to a macroblocl< number variable J 
()>=1) representing the serial number of a macroblock within one of the pictures. Hereinlater, the J-th macroblock In 
the picture is referred to as "MB(j)". 

[0025] In the following step A8, a utilization volume of the capacity of a virtual buffer for I, P or B-pictures, referred 
to as d|(J), dpO) or d^h computed before encoding the macroblock MB(J) as follows: 



Tlx (/-I) 

40 tfi(y)=Q^i(0)+ga-1)- f^j^Q equation (1 8) 

tfp(y)=tfp(0Hg(y-1)- ^f^i^Q equation (19) 



dt{J>d^(OhBU'^ )'^^^^^ equation (20) 

where B(j-1) is the total number of bits generated for encoded macroblocks In the picture up to and including the 
(j-1)th macroblock MB(j-1). NMB is the total number of macroblocks In the picture. d,(j), dp(j), or d^Q) Is the utilization 
volume of the capacity of the virtual buffer at the J-th macroblock MBG) for I, P, or B-picture. 
[0026] d,(0), dp{0), or d^{0) is the initial utilization volume of the virtual buffer for I, P, or B-plcture and given by: 

d/(0)-10xr/31 equation (21) 
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or 



dp(0)^KpX equation (22) 



or 



^b(0)=^b^ equation (23) 

where r is referred to as "reaction parameter" and used for the control of a reaction rate of the feed back loop as 
follows: 

15 r=2x Target_Bitrate/picturejrate equation (24) 

[0027] The final utilization volume of the virtual buffer, referred to as. d,(NI\/IB), dp(NMB), or db(NMB) of the last 
macroblock, i.e., NMB-th macroblock MB(NMB) of the current picture plc(n) will be used as the initial utilization volume 
of the virtual buffer for I, P. or B-picture, I.e., d,(0), dp(0), or d^jo) of the same type to encode the first macroblock MB 
(1) within the next pk^ture pic(n4'1). 
[0028] In the following step A9, the reference quantization parameter Q(|) of the J-th macroblock MB(J) for each of 
the pictures is computed on the basis of the aforesaid utilization volume of the virtual buffer, I.e., d(]) as follows: 

25 QG)=dG)x31/r equation (25) 

[0029] Here, the reference quantization parameter Q(j) is identical with the aforesaid second quantization parameter 
Qj of the j-th macroblock MB(j). 

[0030] In the following step A1 0, the j-th macroblock MBQ) is quantized with the reference quantization parameter 
Q(J) computed in the step A9. In the following step All, the macroblock number variable j is Incremented by one. The 
step All goes forward to the step A1 2 wherein it is judged upon whether the macroblock number variable j is more than 
the total number of macroblocks NIVIB within the n-th picture pic(n) or not. When it is judged that the macroblock 
number variable j Is not more than the total number of macroblocks NMB within the n-th picture pic(n), the step A12 
returns to the step A8. When, on the other hand, it is judged that the macroblock number variable j is more than the 
total number of macroblocks NMB within the n-th picture plc(n), the step A12 goes fonward to the step A13. 
[0031] The macroblock number variable ] thus serves as a loop counter for repeating the process from the steps A8 
to All to encode all the macroblocks from the l^t macroblock MB(1) up to the j-th macroblock MB(j) In the present 
picture pic(n). The entire macroblocks starting from the first macroblock MB(1) up to the NMB-th macroblock MB 
(NMB), in the n-th picture pic(n) can be thus encoded sequentially. 

[0032] In the step A13, the picture number variable n is Incremented by one. Then the step A13 goes forward to the 
step A1 4 wherein it is judged upon whether the picture number variable n is more than the total number of pictures, i, 
e., NPIC or not. When it is judged that the picture number variable n is not more than the total number of pictures, 
NPIC, the step A14 to the step A2. When, on the other hand, it is judged that the picture number variable n is more 
than the total number of pictures, NPIC, this routine of the rate controlling process is temninated. The picture number 
variable n thus serves as a loop counter for repeating the process from steps A2 to A13 to process all the pictures 
from the first picture pic(1) to the n-th picture pic(n) in the present GROUP OF PICTURES. The entire pictures starting 
from the first picture pic(1 ) up to the NPIC-th picture pic(NPIC), In the present GROUP OF PICTURES can be therefore 
processed sequentially. 

[0033] The aforesaid conventional transcoder 50, however, has no infonnation on the structure of GROUP OF PIC- 
TURES such as a picture cycle of I or P-pictures within each of the GROUP OF PICTURES, so that the transcoder 50 
must estimate the structure of GROUP OF PICTURES within the inputted moving picture sequence to allocate the 
number of bits to pictures of each type within the estimated structure of GROUP OF PICTURES. 
[0034] Furthermore, the first conventional transcoder 50 is required to decode the first bits streams almost all 
over the layers such as the sequence layer, the GROUP OF PICTURES layer, the picture layer, the slice layer and the 
macroblock layer in order to derive necessary data for transcoding the first bits streams into the second bit streams 
bj. The operation takes time, thereby causing the delay in the transcoding process. 

[0035] Referring to Fig. 20 of the drawings, there is shown an improvement of the above transcoder 50 as a second 
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conventional transcoder 60. The second conventional transcoder 60 Is adapted to perfonn the rate control without 
estimating the structure of GROUP OF PICTURES. As shown in Fig. 20, the second conventional transcoder 60 com- 
prises a delay circuit 61 and a rate controller 62 In addition to the variable length decoder 51 , the Inverse quantizer 53, 
the quantizer 55 and the variable length encoder 57 same as those of the first conventional transcoder 50 shown In 

s Fig. 18. The same constitutional elements are simply represented by the same reference numerals as those of the 
conventional transcoder 50, and will be thus omitted from description for avoiding tedious repetition. 
[0036] The delay circuit 61 Is Interposed between the variable length decoder 51 and the Inverse quantizer 53 and 
designed to control the flow of the signal from the variable length decoder 51 to the inverse quantizer 53. The delay 
circuit 61 Is operated to delay the operation start time of the inverse quantizer 53 so that the inverse quantizer 53 does 

10 not start the de-quantizing process until the variable length decoder 51 terminates the process of decoding one of the 
pictures In the coded moving picture sequence signal. 

[0037] As shown in Fig. 20, the rate controller 62 of the second conventional transcoder 60 includes a target ratio 
computing unit 63, an input bit summing unit 65, a bit difference computing unit 67, a target output bit updating unit 
69, and a quantization parameter computing unit 71 . 

15 [0038] The target ratio computing unit 63 is electrically connected to the variable length decoder 51 and designed 
to Input an Input bit rate of the first bit streams b^, hereinlater referred to as "Input.Bltrate", from the variable length 
decoder 51 , and Input a target bit rate, hereinlater referred to as "Target^Bltrate" through a temnlnal 83. Alternatively, 
the target bit rate Target_Bitrate may have been stored in an Internal memory, or determined on the basis of internal 
switches. The target ratio computing unit 63 is designed to then compute a target ratio, hereinlater referred to as 

20 "loRatlo" of the target bit rate Target.Bitrate to the input bit rate Input.Bltrate for each of pictures as follows. 

;^o«#;^ Target_Bitrate 

InpuLBitrate equation (26) 

25 [0039] The input bit summing unit 65 Is designed to sum up the number of Inputting bits of the picture decoded by 
the variable length decoder 51 to produce the total number of inputting bits, hereinlater referred to as "T|n". On the 
other hand, the target output bit updating unit 69 is designed to compute a target number of outputting bits to be 
generated by the variable length encoder 57, hereinlater referred to as "Tqu^". The target number of outputting bits 
Tout computed by multiplying the total number of inputting bits T,n by the target ratio ioRatio as follows: 
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''"out=Tjn X ioRatio equation (27) 

[0040] The bit difference computing unit 67 is electrically connected to the variable length encoder 57 and the target 
output bit updating unit 69, and designed to Input a real number of outputting bits encoded by the variable length 
encoder 57, hereinlater referred to as "T^ea,", and input the target number of outputting bits J^^^X' "'"^e bit difference 
computing unit 67 Is designed to then compute a difference between the target number of outputting bits T^ut and the 
real number of outputting bits T^gai* hereinlater referred to as a "difference number of bits", I.e., "T^,,!," as follows: 

Tdiff=Trea|-Toui equation (28) 

[0041 ] The target output bit updating unit 69 is electrically connected to the target ratio computing unit 63, the input 
bit summing unit 65, and the bit difference computing unit 67. The target output bit updating unit 69 Is designed to 
update the target number of outputting bits T^ut on the basis of the difference number of bits T^^^ as follows: 

"^out=Tout-''"diff equation (29) 

[0042] The quantization parameter computing unit 71 is electrically connected to the target output bit updating unit 
69 and designed to compute the reference quantization parameter QQ*) for each of macroblocks MB(j) on the basis of 
the target outputting bits T^ut updated by the target output bit updating unit 69 In accordance with the step II of the TM-5. 
[0043] Fig. 21 shows the flowchart of the rate controlling process perfonned by the above conventional transcoder 
60. The rate controlling process in the transcoder 60 comprises the steps 81 to B13. The steps B6 to B13 are the 
almost same as those of the steps A7 to A14, respectively, in the rate controlling process shown in Fig. 19 except for 
the step 87 wherein the utilization volume of the capacity of the virtual buffer Is computed on the basis of the target 
number of outputting bits Tout Qiven by the target output bit updating unit 69 instead of the target number of bits T,, Tp 
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or Tb computed in the step A3 shown in Fig. 19. The same steps will be thus omitted from description for avoiding 

tedious repetition. 

[0044] in the step B1 , "1" is assigned to the picture number variable n. The step B 1 then goes forward to the step 
B2 wherein the target ratio ioRatIo is computed by the above equation (26). In the following step 83, the difference 
number of bits T^^^ Is computed for the present picture pic(n) by the above equation (28). The step 83 then goes 
forward to the step B4 wherein the number of inputting bits T,„ Is summed up within the first bit streams b-,. in the step 
B5, the target number of outputting bits T^ut is computed by the above equation (27), and further updated by the above 
equation (29). 

[0045] in the second conventional transcoder 60 thus constructed, the inverse quantizer 53, however, cannot start 
the de-quantlzation process until the target transcoding frame is completely decoded, thereby causing the delay In the 

transcoding process. 

[0046] Referring to Figs. 22 and 23 of the drawings, there Is shown another improvement of the above transcoder 
50 as a third conventional transcoder 80. The third conventional transcoder 80 is also adaptable to perfonn the rate 
control without estimating the structure of GROUP OF PICTURES. As shown in Fig. 22, the third conventional trans- 
coder 80 comprises an input temriinal electrically connected to a first transmitting path and designed to input an 
input bit streams bg at the Input bit rate, and an output terminal electrically connected to a second transmitting path 
and designed to output an output bit streams b4 at the target bit rate. In the third conventional transcoder 80, the input 
bit streams bg have a format, non-adaptable for the l\/IPEG-2, different from that of the bit streams b^ of the first and 
second conventional transcoders 50 and 60. The input bit streams have infomriation on the number of coding bits 
previously recorded thereon by the encoder, not shown. 

[0047] The third conventional transcoder 80 comprises a variable length decoder 81 electrically connected to the 
Input temiinal a^, and a rate controller 82 in addition to the inverse quantizer 53, the quantizer 55, and the variable 
length encoder 57 which are same as those of the second transcoder 60 shown in Fig. 20. The rate controller 82 
includes a target output bit updating unit 83, and a quantization parameter computing unit 85 in addition to the target 
ratio computing unit 63, and the bit difference computing unit 67 which are same as those of the second transcoder 
60 shown in Fig. 20. 

[0048] The third conventional transcoder 80 thus constructed can perfomri the rate control on the basis of the forma- 
tion on the number of coding bits previously recorded in the Input bit streams bg. The variable length decoder 81 Is 
adapted to decode the coded moving picture sequence signal within the third bit streams bg to reconstruct the pictures 
and the information on the number of coding bits, and transmit the Information to the inverse quantizer 53. The variable 
length decoder 81 is also adapted to transmit the number of inputting bits Ti^ to the target output bit updating unit 83. 
[0049] The outputting bit updating unit 83 Is designed to compute the target number of outputting bits Tput on the 
basis of the number of inputting bits T,n and the target ratio loRatIo by the above equation (26). The quantization 
parameter computing unit 85 is designed to compute the reference quantization parameter Q(J) of the macroblocks 
MB(J) for each of pictures on the basis of the target number of outputting bits T^^^^ updated by the outputting bit updating 
unit 83 in accordance with the step 11 In the TI\/l-5. The quantizer 55 is then operated to quantize the j-th macroblock 
MB(j) on the basis of the reference quantization parameter Q(j) given by the quantization parameter computing unit 85. 
[0050] Fig. 23 shows the flowchart of the rate controlling process performed by the above third conventional trans- 
coder 80. The rate controlling process In the transcoder 80 comprises the steps CI to CI 3. All the steps C1 to CI 3 
are the same as those of the steps B1 to B1 3, respectively, In the rate controlling process shown in Fig. 21 except for 
the step C4 wherein the number of Inputting bits T,n in the current picture plc(n) is derived from the third bit streams 
bg by the decoder 81 to compute the total number of Inputting bits Tj^. 

[0051] The third conventional transcoder 80 thus constructed has infomriation on the number of coding bits previously 
recorded in the third bits streams bS thereby malting It possible to solve the problem of the delay in the second con- 
ventional transcoder 60. The third conventional transcoder 80, however, has another problem to restrict the form of 
the inputted bit streams. Moreover, the encoder which is linked with the third transcoder 80 must provide with the above 
infonnation on the number of coding bits to be recorded In the bit streams, thereby causing the delay of process in the 
encoder, 

[0052] in any one of the conventional transcoders 50, 60 and 80, the matrix of the de-quantization coefficients de- 
quant is necessary for only the quantizer 55, but unnecessary for the transcoder itself to generate the desired bit 
streams, in order to eliminate the redundant matrix of the de-quantization coefficients dequant, there is proposed a 
fourth conventional transcoder 90 comprising a level converter 91 Instead of the inverse quantizer 53 and the quantizer 
55 of the transcoder 50, as shown In Fig. 26. 

[0053] The level converter 91 is interposed between the variable length decoder 51 and the variable length encoder 
57. The level converter 91 Is designed to input the original picture data for each of pictures. The original picture data 
includes a matrix of original quantization coefficients level for each of macroblocks within the corresponding picture. 
The level converter 91 Is electrically connected to the rate controller 59 and designed to input the second quantization 
parameter Q2 from the rate controller 59. 



9 



10 



IS 



EP1 130 927 A2 

[0054] The level converter 91 Is further designed to convert the original picture data for each of pictures including 
the matrix of original quantization coefficients level into the objective picture data including the nnatrix of re-quantlzation 
coefficients tievel without generating the nnatrlx of the de-quantizatlon coefficients dequant. The following equations 
(30) and (31 ) for the matrix of re-quantlzation coefficients tievel are lead by eliminating the matrix of the de-quantlzation 
coefficients dequant from the above equations (1), (2), (3) and (4). 

tievel = I (level + sign(level) x -1 x — . . .equation (30) 



or 



«e.eA=teve/x^.»i^ equation (31) 

where the above equation (30) is used for the inter macroblock, while the above equation (31) is used for the 
20 intra macroblock. The level converter 91 is thus operable to convert the original picture data, for each of ptetures, into 
the second picture data with the first quantization parameter and the second quantization parameter Qg. The first 
quantization parameter is decoded from the first bit streams by the variable length decoder 51 , while the second 
quantization parameter Q2 is obtained from the rate controller 59. 

[0055] In the fourth conventional transcoder 90, the rate controller 59 is designed to perfomn the rate control over 
25 the encoding process in the transcoder 90 according to the TM-5. The variable length encoder 57 Is electrically con- 
nected to the level converter 91 and to input the above matrix of re-quantlzation coefficients tievel from the level 
converter 91. 

[0056] The fourth conventional transcoder 90 thus constructed can efficiently perfomn the transcoding process at 
high speed without storing the matrix of de-quantlzatlon coefficients dequant in a memory. 

30 [0057] The above conventional transcoders 50, 60, 80 and 90, however, has another problem with the rate-distortion 
perfonnance in converting the quantization level. The detailed description of this problem will be made later. In short, 
the rate-distortion perfomnance in converting the quantization level Is unstable and variable In accordance with the first 
and second quantization parameters and the level of the original quantization coefficients level. Therefore, as the 
reduced information volume becomes larger, the quantization error Is liable to Increase, thereby causing the unstable 

35 rate control in transcoding. 

[0058] The applicant of the present application filed patent applications No. H1 1 -278867 and No. H1 1 -327384. 
[0059] The applicant disclosed apparatus, a method and a computer program product for transcoding a coded moving 
picture sequence, being operable to compute the optimized quantization parameter on the basis of the de-quantizatlon 
parameter and the previously computed quantization parameter in consideration of the characteristic of the rate-dis- 

40 tortion performance dependent on the quantization parameter and the de-quantlzation parameter in the patent appli- 
cation No. H1 1-278867. 

[0060] The transcoder disclosed in the aforesaid patent application No. H1 1-278867, comprising the inverse quan- 
tizer for perfomijng the inverse-quantization operation and the quantizer for perfomning the quantization operation, Is 
characterized in that the transcoder further comprises quantization parameter switching means for switching the quan- 

45 tlzatlon parameter In consideration of the characteristic of the rate-distortion performance dependent on the Inputted 
quantization parameter, thereby making it possible for the transcoder to minimize the quantization en-or occurred when 
the matrix of original quantization coefficients Is transformed to the matrix of re-quantization coefficients. 
[0061 ] The applicant further disclosed apparatus, a method and a computer program product for transcoding a coded 
moving picture sequence, being operable to control the number of reduction-object bits in accordance with the size of 

50 the quantization parameter obtained from the input bit streams In consideration of the number of reduction-object bits 
and the quantization error during the re-quantization operation In the transcoder, thereby enabling to minimize the 
quantization error occurred as a result of the re-quantizatlon operation within the transcoder in the patent application 
No. H 11 -327384. 

[0062] The transcoder disclosed In the aforesaid patent application No. H1 1 -327384, comprises: target reduction bit 
55 number computing means for computing the average number of reduction-object bits regarded as target number of 
reduction-object bits; target bit number computing means for computing the target number of bits on the basis of the 
average number of bits reduced in the Input quantization parameter and the number of bits In the DCT coefficients; 
and quantization scaling factor computing means for specifying the quantization parameter on the basis of the target 
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number of bits computed by the target bit number computing means. The thus constructed transcoder can minimize 
the quantization error occun^ed as a result of the re-quantization operation in the transcoder. Here, bit number is "the 
number of bits". 

[0063] It is, however, unthinkable to deliver information having only video contents when infomiation delivery is done 
as part of business service. Most information would be delivered in the form of multiplexed multimedia streams, i.e., 
MPEG-2 system bit streams, having data information such as video, audio and program information. An apparatus 
operative to convert a bit rate for the l\/IPEG-2 system bit steams, is therefore required. 

[0064] The present invention provides a MPEG-2 system stream transcoder for the MPEG-2 system streams. Fig. 
24 shows renderings of an environment in which the present invention is utilized. 

[0065] Conventional MPEG-2 transport stream rate converters in combination of the prior arts and their problems 
will be epitomized before describing the present invention In detail. 

[0066] As shown in Fig. 25, an apparatus 900 is a simple combination of a MPEG-2 transport stream decoder 910 
and a MPEG-2 transport stream encoder 930. The MPEG-2 transport stream decoder 91 0 comprises a transport stream 
demultiplexer 91 1 , a video decoder 9 13, an audio decoder 91 5, a system information decoder 91 7, and a data dedicated 
decoder 919, The MPEG-2 transport stream encoder 930 comprises a transport stream multiplexer 931 , a video en- 
coder 933, an audio encoder 935, a system information encoder 937, and a data dedicated encoder 939. 
[0067] The apparatus 900 can output MPEG-2 transport streams at a target output bit rate. The apparatus 900, 
however, has another problems resulted from the fact that the video decoder and the video encoder are simply com- 
bined. The problems are as follows. 

(1) Amount of process Is increased. 

The apparatus 900 must perform a series of operations, i.e., decode all the inputted bit streams into pictures, 
and then encode the decoded pictures into appropriate target bit streams. The process of decoding and encoding 
all the inputted bit streams is time consuming. 

(2) Quality of pictures Is deteriorated. 

Once the decoder decodes the inputted bit streams into the pictures, the thus decoded pictures does not 
contain original structure information elements of the Inputted bit streams such as the structure of GROUP OF 
PICTURES and the picture types any more. As a result, the encoder must encode the decoded pictures into the 
target bit streams having structure information elements different from the original structure information elements. 
The B-picture, which is not recurrently referred to, has information volume less than the l-picture and the P-picture, 
which are recurrently referred to so that the quality of pictures as a whole is improved. On the other hand, the B- 
plcture in a frame of the inputted bit streams, for instance, is encoded as the l-picture in the target bit streams, 
thereby causing the quality of pictures to be deteriorated. 

(3) Frame realignment causes delay. 

Once the decoder decodes the inputted bit streams having the B-pictures Into the pictures, the frame sequence 
in the Inputted bits streams Is changed In the pictures. As a result, the encoder must realign the frame sequence 
to encode the decoded pictures into the target bit streams, thereby causing delay. Bit streams in the form of "M=3'* 
type, for instance, cause the delay of three frames to decode the l-picture. and the P-picture before the B-picture 
to realign the frame sequence while being decoded into the pictures, and the thus decoded pictures cause the 
delay of another three frames to encode the l-picture and P-picture before the B-picture to realign the frame se- 
quence while being encoded again. (Here, "M" stands for a cycle of the appearance of the l-picture of the B-picture. 
"M=2" means that one B-picture is inserted between the l-picture or the B-plcture while "M=3" means that two B- 
pictures are inserted between the l-picture or the B-picture. Bit streams are generally In the fonn of "M=3" type.) 
Totally, the delay of six frames is generated in the apparatus 900. 

[0068] In order to solve the aforesaid problems, a rate converter 600 includes a MPEG-2 transport stream demulti- 
plexer, a MPEG-2 video transcoder, and a MPEG-2 multiplexer. 

[0069] The rate converter 600 is shown in Fig. 1 as comprising a MPEG-2 transport stream demultiplexer 610, a 
MPEG-2 multiplexer 620, a MPEG-2 video transcoder 640, and a system controller 650. 

[0070] The rate converter 600 has a MPEG-2 video transcoder interposed between a video bit stream decoder and 
a video bit stream encoder (see Fig. 25) to ensure that the problems (1 ) to (3) are solved with respect to the apparatus 
900 of the simple combination of the MPEG-2 transport stream decoder and the MPEG-2 transport stream encoder. 
[0071 J The audio decoder, the audio encoder, the data dedicated decoder, and the data dedicated encoder are not 
provided in the rate converter 600, which Is operated through a method comprising the steps of: 

a) Inputting MPEG-2 transport streams; 

b) demultiplexing the Inputted MPEG-2 transport streams Into video bit streams and other bit streams such as 
audio, system Infomnatlon and data bit streams; 
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c) compressing only the video bit streams, which has extremely large infomriatlon volume; 

d) modifying just part of fixed codes of the other bit streams if necessary; 

e) multiplexing the compressed video bit streams, and the partly modified other bit streams Into output MPEG-2 
transport streams; and 

f) outputting the output MPEG-2 transport streams. 

[0072] The system controller 650 is operated, Instead of the system Information decoder 91 7 and the system infor- 
mation encoder 937 of the apparatus 900, to modify part of fixed codes of the system information bit streams and 
replace the thus partly modified system infomnation bit streams with the original system infomnation bit streams. 
[0073] The rate converter 600, however, encounters additional problems as follows. 

(1) The bit rate of output MPEG-2 video bit streams is, basically, computed on the basis of the bit rate of output 
MPEG-2 transport streams. The output MPEG-2 transport streams, however, contain other bit streams such as 
audio bit streams and control infomnation bit streams, in addition to the video bit streams. Furthermore, bits such 
as header infomnation bits are generated as a result of packetizing elementary streams and transport streams. 
The bits thus generated are added to the output MPEG-2 transport streams. It is therefore difficult to compute the 
bit rate of output video bit streams. 

(2) The rate converter 600 is not applicable to the output bit rate control for video bit streams in the variable bit 
rate format. 

[0074] The video bit streams contained in the MPEG-2 transport streams are assumed to be In the variable bit rate 
(VBR) format. On the other hand, a CBR rate control method used by a conventional video bit stream transcoder, for 
controlling output bit rate of video bit streams in the constant bit rate (CBR) fonnat on the basis of input bit rate as a 
parameter, or on the basis of the numbers of bits and pictures to be encoded in GOP, and the picture types is not 
applicable to video bit streams in the VBR format. 

[0075] In order to solve the above problems, the present invention is proposed to an apparatus, a method and a 
computer program product for transcoding a coded multiplexed sound and moving picture sequence for MPEG-2 sys- 
tem bit streams, which enable to completely synchronize audio and video bit streams between input and output MPEG- 
2 transport steams on the basis of synchronous infomnation element contained in the input MPEG-2 transport streams; 
and establish a rate control method for controlling output bit rate of video bit streams in the variable bit rate, i.e., VBR 
format. 

[0076] To be solved by the transcoder according to the present invention are problems of: 

(1) completely synchronizing audio and video bit streams between Input and output MPEG-2 transport steams on 
the basis of synchronous infomnation element contained in the input MPEG-2 transport streams; and 

(2) establishing a rate control method for controlling output bit rate of video bit streams in the variable bit rate, i. 
e., VBR format. 

[0077] Required to solve the problem (1) are three conditions as follows: 

Condition (1): Time stamp (PGR) contained in output MPEG-2 transport streams must be set to avalue In acertain 
range, for instance, to the initial value of the input MPEG-2 transport streams, so that the output MPEG-2 transport 
streams do not cause the breakdown of a MPEG-2 decoder buffer when the output MPEG-2 transport streams 
are Inputted to the MPEG-2 decoder 

Condition (2): A video frame of video bit streams and an audio frame of audio bit streams contained in input MPEG- 
2 transport streams must share the same PTSs and DTSs with the same video frame of the video bit streams and 
the same audio frame of the audio bit streams contained in output MPEG-2 transport streams. 
Condition (3): Bit streams constituting a video frame and an audio frame contained in the output MPEG-2 transport 
streams arrive at the decoder at the same time at which bit streams constituting the same video frame and the 
same audio frame contained in the input MPEG-2 transport streams are supposed to arrive at the decoder 

[0078] Provided that only the video bit streams are to be compressed to achieve the target bit rate, the means to 

solve the aforesaid problems will be described hereinlater 

[0079] Condition (1) will be satisfied by a method comprising the steps of: 

(a) decoding a first PGR contained in the inputted MPEG-2 transport streams; 

(b) computing a value of system time clock, i.e., reference synchronous Infomnation element for decoding process, 
hereinlater referred to as "STC". of the first byte of the inputted MPEG-2 transport streams on the basis of the 
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decoded first PGR, the total number of bytes of the MPEG-2 transport streams inputted into the rate converter, 
and an input bit rate; and 

(c) matching the thus computed value of STC of the first byte of the inputted MPEG-2 transport streams with the 
value of STC of the first byte of the outputting MPEG-2 transport streams. 

[0080] Condition (2) will be satisfied by a method comprising the steps of: (a) demultiplexing the inputted MPEG-2 
transport streams into video transport streams and the other bit streams; (b) decoding the video transport streams Into 
video packetlzed elementary streams; (c) decoding the video packetlzed elementary streams Into video elementary 
streams and the corresponding PTS and DTS; (d) storing the PTS and the DTS in a memory unit; and (e) encoding 
video elementary streams and the corresponding PTS and the DTS to generate video packetlzed elementary streams. 
Audio bit streams in the inputted MPEG-2 transport streams remain the same in the outputted MPEG-2 transport 
streams. Accordingly. PTS of the audio bit streams in the inputted MPEG-2 transport streams remains the same in the 
outputted MPEG-2 transport streams. 

[0081] Fig. 3 shows an example of a relationship between the inputted MPEG-2 transport streams and the outputted 
MPEG-2 transport streams, which satisfies the condition (3). Transport stream packets to be reduced, hereinlater 
referred to as "reduction-object transport stream packets", contained in the outputted MPEG-2 transport streams are 
reduced to one third of that of the inputted MPEG-2 transport streams and accordingly, the bit rate of the outputted 
MPEG-2 transport streams is reduced to the half of that of the Inputted MPEG-2 transport streams. This means that 
the transport stream packets to be not reduced, for instance, the bit streams constituting an audio frame are placed 
and interposed between the reduction-object transport stream packets, for instance, the bit streams constituting a video 
frame In accordance with a ratio of output bit rate to Input bit rate, thereby making it possible that the bit streams 
constituting a video frame and an audio frame contained in the output MPEG-2 transport streams arrives at the decoder 
at the same time at which the bit streams constituting the same video frame and the same audio frame contained in 
the input MPEG-2 transport streams are supposed to arrive at. For avoiding tedious repetition, transport stream packets 
will be referred to as TS packets, hereinlater. 

[0082] Here, reduction-object TS packets are all the TS packets excluding two types of packets consisting of transport 
packets including video bit streams contained in the inputted MPEG-2 transport stream, andTS packets apt to change 
in accordance with the control state in the system such as PAT and PMT, which will be described hereinlater. 
[0083] Furthermore, another conditions must be satisfied to reduce the number of bits of the reduction-object video 
bit streams to decrease the bit rate. 

[0084] Fig. 4 shows examples of the transition of utilization volume of a Video Buffering Verifier buffer, hereinlater 
refen^ed to as "VBV buffer" (Video Buffering Verifier: a parameter indicative of the size of the virtual buffer used for 
controlling the number of generated bits) and DTS (Decoding Time Stamp: decoding time management information) 
while the bit rate of video bit streams contained in the outputted MPEG-2 transport streams is reduced to the half of 
that of video bit streams contained in the inputted MPEG-2 transport streams in the case of (a) and the case of (b). 
[0085] In the case of (a), the number of bits of l-pictures are not reduced while the number of bits of P-pictures and 
B-pictures are reduced. In the case of (b), on the other hand, l-pictures, P-pictures, and B-pictures are evenly reduced. 
This means that the ratio of l-pictures, P-pictures, and B-pictures in the input video bit streams remains the same as 
that In the output video bit streams. 

[0086] In Fig. 4, the upper graph shows the transition of the video elementary streams contained in the inputted bit 
streams before the transcoding process, and the lower graph shows the transition of the video elementary streams 
contained in the outputted bit streams after the transcoding process. "B" Indicates the size of a receiving buffer, "B 
(n)*" Indicates the VBV buffer utilization volume just before the n-th picture is decoded, and "B(n)" indicates the VBV 
buffer utilization volume just after the n-th picture is decoded and the number of bits for the size of one frame Is removed 
from the buffer. The VBV buffer utilization volume must fluctuate in a range of 0 and B. The slope of a line segment 
indicates a bit rate. 

[0087] The receiving buffer waits until the time indicated by DTS, and starts decoding a frame consisting of Input 
video elementary streams when the time indicated by DTS elapses. On the other hand, the receiving buffer waits until 
the time indicated by DTS', and start decoding the frame consisting of the output video elementary streams when the 
time indicated by DTS' elapses. 

[0088] In the case of (a), DTS' is greater than DTS. This means that some of the video elementary streams constituting 
the frame may not arrive at the MPEG-2 transport stream decoder until the time indicated by the DTS, thereby falling 
to meet the condition (3). In other words, the output video elementary streams cannot have the DTS of the input video 
elementary streams as start time of decoding the frame consisting of the video elementary streams so as to meet the 
condition (2). 

[0089] In the case of (b), DTS' is equal to DTS. This means that all of the video elementary streams constituting the 
frame will arrive at the M PEG-2 transport stream decoder until the time Indicated by the DTS, thereby making it possible 
to start decoding the frame at the time indicated by the DTS, and meet the condition (3). Therefore, the rate control 
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method must satisfy the condition of DTS' =DTS so as to reduce the number of bits of the video bit streams to reduce 
the bit rate. 

[0090] A rate control method to solve the problem (2) will be described hereinlater. 

[0091] Fig. 5 shows a schematic block diagram describing the concept of the rate control method for controlling 
output bit rate of the video bit streams in the VBR format performed during the transcoding process. In Fig. 5, the input 
MPEG-2 transport streams at an input bit rate is converted Into the output MPEG-2 transport streams at the target bit 
rate, which is the half of the input bit rate. 

[0092] Each of the MPEG-2 transport stream consists of packets of 1 88 bytes. The number of packets to be inputted 
at time Interval of a predetermined duration is always the same. This means that the number of packets inputted at 
time interval of a predetennined duration can be computed on the basis of the product of the input bit rate and the 
predetermined time. This leads to the fact that a target number of packets to be outputted at time interval of the pre- 
detemrilned duration can be computed In the similar manner to the number of packets to be inputted. 
[0093] Furthermore, the number of video transport streams and the number of transport streams to be not reduced, 
hereiniater referred to as "non-reduction transport streams", contained in the input MPEG-2 transport streams can be 
computed on the basis of information acquired when the input MPEG-2 transport steams are demultiplexed. Provided 
that the non-reduction transport streams are not to be compressed, the target number of output video transport stream 
packets at time interval of a predetennined duration (n) can be computed by subtraction of the number of packets of 
the non-reduction transport streams and the number of PAT and PMT packets to be outputted at time interval of the 
predetermined duration (n) from the target number of total packets to be outputted at time interval of the predetermined 
duration (n). 

[0094] As will be understood from the foregoing descriptions, the rate control for controlling output bit rate of video 
bit streams in the VBR format can be performed basically through the steps of: (a) demultiplexing the MPEG-2 transport 
streams into video transport streams and other transport streams such as non-reduction transport streams; (b) com- 
puting the number of video transport streams and the number of non-reduction transport streams; (c) computing the 
target bit rate, i.e., the target number of output video transport stream packets to be outputted at time interval of a 
predetermined duration (n), by subtraction of the number of the non-reduction transport stream packets and the number 
of the PAT and PMT packets to be outputted at time interval of the predetennined duration (n) from the target number 
of total packets to be outputted at time interval of the predetennined duration (n); and (d) converting the video transport 
streams Into the output video elementary streams having the target number of the output video transport stream pack- 
ets, thereby making it possible to control the output bit rate less than the input bit rate. 

[0095] There is another problem which makes It difficult to detennine the target number of output video transport 
stream packets. While transcoding the MPEG-2 transport streams, video elementary streams are converted to PES 
(PES: Packetlzed Elementary Stream) packets and TS (TS: Transport Stream) packet, new packet headers are gen- 
erated and attached to the PES packets and TS packets, thereby causing overhead. The exact number of packet 
headers thus newly generated, however, cannot be estimated beforehand. The target bit number of output video ele- 
mentary streams to be outputted at time interval of the predetermined duration (n), therefore, is determined through 
the steps of: (a) computing the overhead generated through the process of converting the video elementary streams 
to PES packets and TS packets prior to the time interval of the predetennined duration (n-1); and (b) computing the 
target bit number of the output video elementary streams at time interval of the predetennined duration (n) In consid- 
eration of the overhead computed in the step (a). 

[0096] The above described rate control method will satisfy the condition (3) of the problem (1 ). 
SUMMARY OF THE INVENTION 

[0097] It is, therefore, an object of the present Invention to provide a method of transcoding a coded multiplexed 
sound and moving picture sequence, which sets the time stamp (PGR) contained in output MPEG-2 transport streams 
to a value in a certain range, for instance, to the Initial value of the input MPEG-2 transport streams, so that the output 
MPEG-2 transport streams do not cause the breakdown of a MPEG-2 decoder buffer when the output MPEG-2 transport 
streams are inputted to the MPEG-2 decoder. 

[0098] It is another object of the present invention to provide a method of transcoding a coded multiplexed sound 
and moving picture sequence, which makes it possible for a video frame of video bit streams and an audio frame of 
audio bit streams contained in input MPEG-2 transport streams to share the same PTSs and DTSs with the same video 
frame of the video bit streams and the same audio frame of the audio bit streams contained In output MPEG-2 transport 
streams. 

[0099] It is a further object of the present invention to provide a method of transcoding a coded multiplexed sound 
and moving picture sequence, which makes It possible for bit streams constituting a video frame and an audio frame 
contained in the output MPEG-2 transport streams to arrive at the decoder at the same time at which bit streams 
constituting the same video frame and the same audio frame contained in the Input MPEG-2 transport streams are 
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supposed to arrive at. 

[0100] It is a still further object of the present Invention to provide an apparatus of transcoding a coded multiplexed 
sound and moving picture sequence, which sets the time stamp (PCR) contained in output MPEG-2 transport streams 
to a value In a certain range, for instance, to the initial value of the input MPEG-2 transport streams, so that the output 
5 MPEG-2 transport streams do not cause the breakdown of a MPEG-2 decoder buffer when the output MPEG-2 transport 
streams are inputted to the MPEG-2 decoder. 

[0101] It is a yet further object of the present invention to provide an apparatus of transcoding a coded multiplexed 
sound and moving picture sequence, which makes It possible for a video frame of video bit streams and an audio frame 
of audio bit streams contained in input MPEG-2 transport streams to share the same PTSs and DTSs with the same 
10 video frame of the video bit streams and the same audio frame of the audio bit streams contained in output MPEG-2 
transport streams. 

[0102] It Is a yet further object of the present invention to provide an apparatus of transcoding a coded multiplexed 
sound and moving picture sequence, which makes it possible for bit streams constituting a video frame and an audio 
frame contained in the output MPEG-2 transport streams to arrive at the decoder at the same time at which bit streams 
^5 constituting the same video frame and the same audio frame contained in the input MPEG-2 transport streams are 
supposed to arrive at. 

[0103] It is a yet further object of the present Invention to provide a computer program for transcoding a coded 
multiplexed sound and moving picture sequence, which sets the time stamp (PCR) contained in output MPEG-2 trans- 
port streams to a value in a certain range, for Instance, to the initial value of the input MPEG-2 transport streams, so 
20 that the output MPEG-2 transport streams do not cause the breakdown of a MPEG-2 decoder buffer when the output 
MPEG-2 transport streams are inputted to the MPEG-2 decoder. 

[0104] It is a yet further object of the present invention to provide a computer program for transcoding a coded 
multiplexed sound and moving picture sequence, which makes it possible for a video frame of video bit streams and 
an audio frame of audio bit streams contained in input MPEG-2 transport streams to share the same PTSs and DTSs 
25 with the same video frame of the video bit streams and the same audio frame of the audio bit streams contained in 
output MPEG-2 transport streams. 

[0105] It is a yet further object of the present Invention to provide a computer program for transcoding a coded 
multiplexed sound and moving picture sequence, which makes it possible for bit streams constituting a video frame 
and an audio frame contained in the output MPEG-2 transport streams to arrive at the decoder at the same time at 
30 which bit streams constituting the same video frame and the same audio frame contained In the input MPEG-2 transport 
streams are supposed to arrive at. 

[01 06] In accordance with a fist aspect of the present invention, there is provided a method of transcoding a coded 
multiplexed sound and moving picture sequence, comprising the steps of: 

35 (a) inputting a first coded signal through a first transmitting path at an input bit rate; 

(b) demultiplexing the first coded signal Inputted in the Inputting step (a) into one or more first data strings, one or 
more second data strings, and one or more third data strings, the one or more first data strings each having a 
number of real inputting bits; 

(c) transfonning the one or more first data strings demultiplexed in the demultiplexing step (b) into one or more 
40 first data strings having a number of real outputting bits less than the number of real inputting bits of the one or 

more first data strings, respectively; 

(d) multiplexing the one or more transformed first data strings transformed in the transforming step (c), the one or 
more second data strings demultiplexed In the demultiplexing step (b), and one or more corrected third data strings 
to generate a second coded signal; 

^5 (e) correcting the one or more third data strings demultiplexed in the demultiplexing step (b), on the basis of the 

first coded signal, In accordance with a change of the second coded signal to generate the one or more corrected 
third data strings having a number of real inputting bits when the one or more transformed first data strings, the 
one or more second data, and the one or more corrected third data strings are multiplexed in the multiplexing step 
(d); and 

so (f) outputting the second coded signal through a second transmitting path at a target bit rate lower than the Input 

bit rate of the first coded signal. 

[0107] In the aforesaid method, the inputting step (a) may have the step of (a2) inputting MPEG-2 transport streams. 
Furthermore, the demultiplexing step (b) may have the step of (b2) demultiplexing the MPEG-2 transport streams 
55 inputted in the Inputting step {a2) into one or more transport stream packets having a coded video signal having a 
number of real Inputting bits as one or more first data strings. The outputting step (f) may have the step of (f2) outputting 
MPEG-2 transport streams having a video signal having a number of real outputting bits less than the number of real 
inputting bits of the coded video signal. 
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[0108] Alternatively, in the aforesaid method, the Inputting step (a) has the step of (a3) inputting coded multiplexed 
sound and moving picture sequence streams. The demultiplexing step (b) may have the step of (b3) demultiplexing 
the coded multiplexed sound and moving picture sequence streams Inputted In the Inputting step (a3) into one or more 
transport stream packets having a coded moving picture sequence signal having a number of real inputting bits as the 
one or more first data strings. Furthermore, the outputting step (f) may have the step of (fS) outputting coded multiplexed 
sound and moving picture sequence streams having a coded moving picture sequence signal having a number of real 
outputting bits less than the number of real Inputting bits of the coded moving picture sequence signal. 
[01 09] Alternatively, the aforesaid method may further comprise the steps of: (g) computing a value of system clock 
indicative of a start time of the demultiplexing step (b), on the basis of a first reference time information element con- 
tained In the first coded signal; and (h) computing an Initial value of the system clock for the second coded signal on 
the basis of the value of the system clock computed in the computing step (g). 

[0110] Alternatively, in the aforesaid method, the transforming step (c) may further comprise the steps of: (c51) 
decoding the one or more transport stream packets having a coded video signal having a number of real Inputting bits 
to reconstruct and output video PES packets; (c52) decoding the video PES packets decoded In the decoding step 
(c51) to reconstruct and output video elementary streams having a real inputting bits, decoding time management 
Infonnation element DTS and presentation time management infomiation element PTS of the video elementary 
streams, and PTS_DTS flags Indicative of presence of the decoding time management Infomnatlon element DTS and 
the presentation time management information element PTS; (c53) transforming the video elementary streams decod- 
ed and outputted in the decoding step (c52) into video elementary streams having a number of real outputting bits less 
than the number of real inputting bits of the video elementary streams; (c54) generating transfomned video PES packets 
on the basis of the transformed video elementary stream transfomned in the transfomning step (c53), the decoding time 
management infomiation element DTS, the presentation time management information element PTS and the PTS_DTS 
flags indicative of presence of the decoding time management Information element DTS and the presentation time 
management information element PTS decoded in the decoding step (c52); and (c55) encoding the transfomied video 
PES packets generated in the generating step (c54) to generate a transfomned transport stream packets having a 
number of real outputting bits less than the number of real inputting bits of the transport stream packets. 
[0111] Altematively, in the aforesaid method, the demultiplexing step (b) may have the step of demultiplexing the 
first coded signal into transport stream packets having a coded audio signal as the one or more second data strings. 
[01 1 2] Altematively, the demultiplexing step (b) may have the step of (b71 ) demultiplexing MPEG-2 transport streams 
inputted at a predetermined time interval In the inputting step (a) into the one or more first data strings, the one or more 
second data strings, and the one ormore third data strings, each having a number of real Inputting bits. Thetransfonning 
step (c) may comprise the steps of: (c71) decoding the one or more first data strings to reconstruct video elementary 
streams having a number of real inputting bits and other information elementary streams, and separating the video 
elementary streams from the other information elementary streams; (c72) transforming the video elementary streams 
reconstructed and separated in the decoding step (c71 ) to generate output video elementary streanns having a number 
of real outputting bits less than the number of real inputting bits of the video elementary streams; and (c73) generating 
one or more transfomned first data strings having a number of real outputting bits less than the number of real inputting 
bits of the one or more first data strings demultiplexed In the demultiplexing step (b71 ) on the basis of the output video 
elementary streams generated in the transforming step (c72) and the other inf omnation elementary steams reconstruct- 
ed and separated in the decoding step (c71). Also, the multiplexing step (d) may have the step of (d71) multiplexing 
the one or more transformed first data strings generated in the generating step (c73), the one or more second data 
strings demultiplexed in the demultiplexing step (b71), and the one or more corrected third data strings corrected in 
the con-ecting step (e) to generate a second coded signal to be outputted at the predetermined time interval. 
[0113] Alternatively In the aforesaid method, the transfomning step (c) may comprise the steps of: (c81) assuming 
that a number of real outputting bits of the one or more second data strings contained in the second coded signal at 
the predetermined time Interval Is equal to a number of real Inputting bits of the one or more second data strings 
contained in the first coded signal at the predetermined time interval; (c82) assuming that a number of real inputting 
bits of the one or more third data strings contained in the second coded signal at the predetennined time interval is 
equal to the number of real inputting bits of the one or more corrected third data strings generated in the con-ecting 
step at the predetemnined time interval (e); (c83) subtracting the number of real inputting bits of the one or more second 
data strings contained in the first coded signal at the predetermined time interval and the number of real inputting bits 
of the one or more third data strings contained In the second coded signal at the predetennined time Interval from a 
target number of outputting bits of all data strings contained in the second coded signal at the predetennined time 
Interval to generate a value A; (c84) subtracting a total number of real outputting bits of one or more transformed first 
data strings generated In the generating step (c73) prior to the predetermined time Interval from a total target number 
of outputting bits of one or more transfomned first data strings generated in the generating step (c73) prior to the 
predetennined time interval to generate a value B; and (c85) computing a target number of outputting bits of the trans- 
formed first data strings generated at the predetennined time interval in the generating step (c73) by adding the value 
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A and the value B. 

[0114] Alternatively, in the aforesaid method, the transfomning step (c72) connprises the steps of: (c91) computing a 
target number of outputting bits of the output video elementary streams on the basis of outputting bits of the second 
coded signal able to be outputted at the predetemnined time interval; (c92) computing a reference ratio of outputting 

5 bits to inputting bits on the basis of the target number of outputting bits of the output video elementary streams computed 
In the computing step {c91) and the real inputting bits of the video elementary streams reconstructed in the decoding 
step (c71 ); and (c93) computing a quantization scaling factor required for transforming the video elementary streams 
to generate the output video elementary streams, on the basis of the reference ratio of outputting bits to Inputting bits 
computed in the computing step (c92). 

10 [0115] Altematively, In the aforesaid method, the computing step (c91) comprises the steps of: (c101) computing a 
ratio of a total number of real outputting bits of the first one or more transfonned data strings generated in the generating 
step (c73) prior to the predetennined time interval to a total number of real outputting bits of the output video elementary 
streams prior to the predetenmlned time Interval; and (c1 02) computing a target number of outputting bits of the output 
video elementary streams at the predetermined time interval on the basis of the ratio computed in the computing step 

IS (c1 01 ), and the computing step (c92) has the step of (c1 03) computing a reference ratio of outputting bits to inputting 
bits on the basis of the target number of outputting bits of the output video elementary streams computed in the com- 
puting step (c102) and the real inputting bits of the video elementary streams decoded in the decoding step (c71). 
[0116] Altematively, in the aforesaid method, the transforming step (c) comprises the steps of: (c1 101) computing a 
total number of real outputting bits of the output video elementary streams generated in the transforming step (c72); 

20 (c1 1 02) computing a sum of a target number of outputting bits of the output video elementary streams at the predeter- 
mined time Interval and a total number of real outputting bits of the output video elementary streams which have been 
generated until the time when video elementary streams inputted prior to the predetermined time interval into a video 
ES buffer are consumed; (c1103) judging upon whether the total number of real outputting bits of the output video 
elementary streams computed in the computing step (c1101) is greater than the sum computed in the computing step 

25 (c1 1 02); and (c1 1 04) terminating the transforming step (c) and starting the multiplexing step (d) for processing the one 
or more transformed first data strings when it is judged that the sum is greater than the total number of real outputting 
bits in the judging step (c1103). 

[0117] Alternatively, in the aforesaid method, the multiplexing step (d) may comprise the steps of: computing a dif- 
ference by subtracting a value of lastly past presentation time management information element PTS of the first coded 

30 signal from a value of a synchronous time Information element PCR located In a head position of a data string of the 
one or more second data strings; (d1202) computing a difference between passing time of the data string of the one 
or more second data strings in the first coded signal and passing time of the data string of the one or more second 
data strings in the second coded signal; (d1 203) judging upon whether the difference computed in the computing step 
(d1201) is smaller than the difference computed in the computing step (d1202); and (d1204) locating the data string 

35 of the one or more second data strings in a rearward position of a data string of the one or more first data strings to 
be located in a rearward position of the data string of the one or more second data strings when it is judged that the 
difference computed in the computing step (d1201) is smaller than the difference computed in the computing step 
(d1202) in the judging step (d1203), and the outpuning step (f) has the step of (f1201) outputting the second coded 
signal at the predetemnined time interval. 

40 [01 1 8] Alternatively, In the aforesaid method, the multiplexing step (d) may comprise the steps of: (d1 301 ) computing 
a difference by subtracting a value of a synchronous time infonmation element PCR to be past subsequently in the first 
coded signal from a value of a presentation time management infonnation element PTS of a just past data string of 
the one or more second data strings; (d1302) computing a difference between passing time of the data string of the 
one or more second data strings in the first coded signal and passing time of the data string of the one or more second 

45 data strings in the second coded signal; (d1303) judging upon whether the difference computed In the computing step 
(d1301) is smaller than the difference computed in the computing step (d1302); and (d1304) locating the data string 
of the one or more second data strings in a forward position of a data string of the one or more first data strings to be 
located in a fonward position of the data string of the one or more second data strings when it is judged that the difference 
computed in the computing step (d1 301 ) Is smaller than the difference computed in the computing step (d1 302) in the 

50 judging step (d1303), and the outputting step (f) has the step of (f1301) outputting the second coded signal at the 
predetermined time interval. 

[01 19] In accordance with a second aspect of the present Invention, there Is provided an apparatus of transcoding 
a coded multiplexed sound and moving picture sequence, comprising: inputting means for inputting a first coded signal 
through a first transmitting path at an input bit rate; demultiplexing means for demultiplexing the first coded signal 
55 inputted by the inputting means into one or more first data strings, one or more second data strings, and one or more 
third data strings, the one or more first data strings each having a number of real inputting bits; transfonmlng means 
for transfomriing the one or more first data strings demultiplexed by the demultiplexing means Into one or more first 
data strings having a number of real outputting bits less than the number of real inputting bits of the one or more first 
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data strings, respectively; multiplexing means formultlplexing the one or more transformed first data strings transformed 
by the transfomiing means, the one or more second data strings demultiplexed by the demultiplexing means, and one 
or more corrected third data strings to generate a second coded signal; correcting means for correcting the one or 
more third data strings demultiplexed by the demultiplexing means, on the basis of the first coded signal. In accordance 
s with a change of the second coded signal to generate the one or more corrected third data strings having a number 
of real inputting bits when the one or more transfomied first data strings, the one or more second data, and the one 
or more corrected third data strings are multiplexed by the multiplexing means; and outputting means for outputting 
the second coded signal through a second transmitting path at a target bit rate lower than the input bit rate of the first 
coded signal. 

10 [01 20] In the aforesaid apparatus, the Inputting means for inputting a first coded signal through a first transmitting 
path may be operable to input MPEG-2 transport streams, the demultiplexing means may be operable to demultiplex 
the MPEG-2 transport streams inputted by the Inputting means Into one or more transport stream packets having a 
coded video signal having a number of real inputting bits as one or more first data strings, and the outputting means 
may be operable to output MPEG-2 transport streams having a video signal having a number of real outputting bits 

15 less than the number of real Inputting bits of the coded video signal. 

[0121 J Alternatively, In the aforesaid apparatus, the inputting means for Inputting a first coded signal through a first 
transmitting path Is operable to Input coded multiplexed sound and moving picture sequence streams, the demultiplex- 
ing means is operable to demultiplex the coded multiplexed sound and moving picture sequence streams inputted by 
the inputting means Into one or more transport stream packets having a coded moving picture sequence signal having 

20 a number of real Inputting bits as the one or more first data strings, and the outputting means is operable to output 
coded multiplexed sound and moving picture sequence streams having a coded moving picture sequence signal having 
a number of real outputting bits less than the number of real Inputting bits of the coded moving picture sequence signal. 
[0122] Alternatively, the aforesaid apparatus may further comprise reference time setting means for computing a 
value of system clock Indicative of a start time of the demultiplexing means, on the basis of a first reference time 

25 infomnation element contained in the first coded signal, and compute an initial value of the system clock forthe second 
coded signal on the basis of the value of the system clock. 

[0123] Alternatively, In the aforesaid apparatus the transfomning means may further comprise: a video transport 
stream packet decoding unit for decoding the one or more transport stream packets having a coded video signal having 
a number of real Inputting bits to reconstruct and output video PES packets; a video PES packet decoding unit for 

30 decoding the video PES packets decoded by the video transport stream packet decoding unit to reconstruct and output 
video elementary streams having a real inputting bits, decoding time management information element DTS and pres- 
entation time management information element PTS of the video elementary streams, and PTS_DTS flags indicative 
of presence of the decoding time management infonnation element DTS and the presentation time management in- 
formation element PTS; a transforming unit for transfomning the video elementary streams decoded and outputted by 

35 the video PES packet decoding unit into video elementary streams having a number of real outputting bits less than 
the number of real inputting bits of the video elementary streams; a video PES packet generating unit for generating 
transformed video PES packets on the basis of the transformed video elementary stream transformed by the trans- 
forming unit, the decoding time management infonnation element DTS, the presentation time management infonnation 
element PTS and the PTS_DTS flags Indicative of presence of the decoding time management information element 

40 DTS and the presentation time management Information element PTS decoded by the video PES packet decoding 
unit; and a video transport stream packet generating unit for encoding the transfomned video PES packets generated 
by the video PES packet generating unit to generate a transfonned transport stream packets having a number of real 
outputting bits less than the number of real inputting bits of the transport stream packets. 

[0124] Alternatively, in the aforesaid apparatus, the demultiplexing means may be operative to demultiplex the first 
45 coded signal into transport stream packets having a coded audio signal as the one or more second data strings. 

[0125] Alternatively, In the aforesaid apparatus, the demultiplexing means may be operative to demultiplex MPEG- 
2 transport streams inputted at a predetemriined time inten/al by the inputting means into the one or more first data 
strings, the one or more second data strings, and the one or more third data strings, each having a number of real 
inputting bits, and the transforming means may comprise: a video elementary stream decoding unit for decoding the 
50 one or more first data strings to reconstruct video elementary streams having a number of real inputting bits and other 
information elementary streams, and separating the video elementary streams from the other Information elementary 
streams; a video elementary stream transfomriing unit for transforming the video elementary streams reconstructed 
and separated by the video elementary stream decoding unit to generate output video elementary streams having a 
number of real outputting bits less than the number of real inputting bits of the video elementary streams; and a trans- 
55 formed first data string generating unit for generating one or more transfonned first data strings having a number of 
real outputting bits less than the number of real inputting bits of the one or more first data strings demultiplexed by the 
demultiplexing means on the basis of the output video elementary streams generated by the video elementary stream 
transfomriing unit and the other infonnation elementary steams reconstructed and separated by the video elementary 
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Stream decoding unit, whereby the multiplexing means Is operable to multiplex the one or more transformed first data 
strings generated by the transf onned first data string generating unit, the one or more second data strings demultiplexed 
by the demultiplexing means, and the one or more corrected third data strings corrected by the correcting means to 
generate a second coded signal to be outputted at the predetemnined time Interval. 

5 [01 26] Alternatively, In the aforesaid apparatus may further comprise a computing unit (A) being operative to: assume 
that a number of real outputting bits of the one or more second data strings contained in the second coded signal at 
the predetemnined time interval is equal to a number of real inputting bits of the one or more second data strings 
contained In the first coded signal at the predetemnined time interval; assume that a number of real inputting bits of 
the one or more third data strings contained in the second coded signal at the predetemnined time interval is equal to 

10 the number of real inputting bits of the one or more corrected third data strings generated by the correcting means at 
the predetermined time Interval; subtract the number of real Inputting bits of the one or more second data strings 
contained in the first coded signal at the predetermined time Interval and the number of real inputting bits of the one 
or more third data strings contained in the second coded signal at the predetenninedtime interval from a target number 
of outputting bits of all data strings contained In the second coded signal at the predetemnined time interval to generate 

*5 a value A; subtract a total number of real outputting bits of one or more transformed first data strings generated by the 
transformed first data string generating unit prior to the predetermined time interval from a total target number of out- 
putting bits of one or more transfonned first data strings generated by the transformed first data string generating unit 
prior to the predetermined time interval to generate a value B; and compute a target number of outputting bits of the 
transformed first data strings generated at the predetennined time interval by the transfomned first data string generating 

20 unit by adding the value A and the value B. 

[0127] Alternatively, In the aforesaid apparatus may comprise: computing unit (B) being operative to compute a target 
number of outputting bits of the output video elementary streams on the basis of outputting bits of the second coded 
signal able to be outputted at the predetermined time interval; computing unit (C) being operative to compute a reference 
ratio of outputting bits to inputting bits on the basis of the target number of outputting bits of the output video elementary 

25 streams computed and the real inputting bits of the video elementary streams reconstructed by the video elementary 
stream decoding unit; and computing unit (D) being operative to compute a quantization scaling factor required for 
transfomilng the video elementary streams to generate the output video elementary streams on the basis of the ref- 
erence ratio of outputting bits to Inputting bits. 

[0128] Alternatively, in the aforesaid apparatus, which may further comprise: computing unit (E) being operative to 
30 compute a ratio of a total number of real outputting bits of the first one or more transformed data strings generated by 
the transformed first data string generating unit prior to the predetermined time Interval to a total number of real out- 
putting bits of the output video elementary streams prior to the predetemnined time interval; computing unit (F) being 
operative to compute a target number of outputting bits of the output video elementary streams at the predetennined 
time interval on the basis of the ratio computed by the computing unit (E); and computing unit (G) being operative to 
35 compute a reference ratio of outputting bits to inputting bits on the basis of the target number of outputting bits of the 
output video elementary streams and the real Inputting bits of the video elementary streams decoded by the video 
elementary stream decoding unit. 

[0129] Altematively, in the aforesaid apparatus, the transf omning means may comprise: computing unit (H) being 
operative to compute a total number of real outputting bits of the output video elementary streams generated by the 

40 video elementary stream transforming unit; computing unit (I) being operative to compute a sum of a target number of 
outputting bits of the output video elementary streams at the predetemnined time Interval and a total number of real 
outputting bits of the output video elementary streams which have been generated until the time when video elementary 
streams inputted prior to the predetennined time interval into a video ES buffer are consumed; judging unit (A) being 
operative to judge upon whether the total number of real outputting bits of the output video elementary streams com- 

45 puted by the computing unit (H) is greater than the sum computed by the computing unit (I); and control unit being 
operative to temninate the transfomning means and starting the multiplexing means for processing the one or more 
transformed first data strings when It is judged that the sum is greater than the total number of real outputting bits by 
the judging unit (A). 

[0130] Altematively, in the aforesaid apparatus, the multiplexing means may comprise: computing unit (J) being 
50 operative to compute a PTS to PCR difference by subtracting a value of lastly past presentation time management 
Infomnatlon element PTS of the first coded signal from a value of a synchronous time information element PCR located 
in a head position of a data string of the one or more second data strings; computing unit (K) being operative to compute 
a passing time difference between passing time of the data string of the one or more second data strings In the first 
coded signal and passing time of the data string of the one or more second data strings In the second coded signal; 
55 judging unit (B) being operative to judge upon whether the PTS to PCR difference computed by the computing unit (J) 
is smaller than the passing time difference computed by the computing unit (K); and locating unit (A) being operative 
to locate the data string of the one or more second data strings in a rearward position of a data string of the one or 
more first data strings to be located in a rearward position of the data string of the one or more second data strings 
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when it is judged tat the PTS to PCR difference computed by the computing unit (J) is smaller than the passing time 
difference computed by the computing unit (K) by the judging unit (B), and the outputting means Is operative to output 
the second coded signal at the predetermined time interval. 

[0131] Alternatively, in the aforesaid apparatus, the multiplexing means may comprise: computing unit (L) being 
operative to compute a PCR to PTS difference by subtracting a value of a synchronous time information element PCR 
to be past subsequently in the first coded signal from a value of a presentation time management infomnation element 
PTS of a just past data string of the one or more second data strings; computing unit (M) being operative to compute 
a passing time difference between passing time of the data string of the one or more second data strings in the first 
coded signal and passing time of the data string of the one or more second data strings in the second coded signal; 
judging unit (0) being operative to judge upon whether the PCR to PTS difference computed by the computing unit (L) 
is smaller than the passing time difference computed by the computing unit (M); and locating unit (B) being operative 
to locate the data string of the one or more second data strings in a forward position of a data string of the one or more 
first data strings to be located in a forward position of the data string of the one or more second data strings when it is 
judged that the PCR to PTS difference computed by the computing unit (L) is smaller than the passing time difference 
computed by the computing unit (M), and the outputting means may be operative to output the second coded signal 
at the predetennined time interval. 

[0132] In accordance with the third aspect of the present invention, there is provided a computer program product 
comprising a computer usable storage medium having computer readable code embodied therein for transcoding a 
coded multiplexed sound and moving picture sequence, comprising: (a) computer readable program code for inputting 
a first coded signal through a first transmitting path at an Input bit rate; (b) computer readable program code for de- 
multiplexing the. first coded signal inputted by the computer readable program code (a) into one or more first data 
strings, one or more second data strings, and one or more third data strings, the one or more first data strings each 
having a number of real inputting bits; (c) computer readable program code for transforming the one or more first data 
strings demultiplexed by the computer readable program code (b) into one or more first data strings having a number 
of real outputting bits less than the number of real inputting bits of the one or more first data strings, respectively; (d) 
computer readable program code for multiplexing the one or more transfomned first data strings transformed by the 
computer readable program code (c), the one or more second data strings demultiplexed by the computer readable 
program code (b), and one or more corrected third data strings to generate a second coded signal; (e) computer 
readable program code for correcting the one or more third data strings demultiplexed by the computer readable pro- 
gram code (b), on the basis of the first coded signal, in accordance with a change of the second coded signal to generate 
the one or more corrected third data strings having a number of real inputting bits when the one or more transformed 
first data strings, the one or more second data, and the one or more corrected third data strings are multiplexed by the 
computer readable program code (d); and (f) computer readable program code for outputting the second coded signal 
through a second transmitting path at a target bit rate lower than the input bit rate of the first coded signal, 
[0133] Alternatively, in the aforesaid computer program product, the computer readable program code (a) may have 
computer readable program code (a2) for inputting MPEG-2 transport streams. The computer readable program code 
(b) may have computer readable program code (b2) for demultiplexing the MPEG-2 transport streams inputted by the 
computer readable program code (a2) into one or more transport stream packets having a coded video signal having 
a number of real inputting bits as one or more first data strings. Also, the computer readable program code (f) may 
have computer readable program code (f2) for outputting MPEG-2 transport streams having a video signal having a 
number of real outputting bits less than the number of real inputting bits of the coded video signal. 
[0134] Alternatively, in the aforesaid computer program product, the computer readable program code (a) may have 
computer readable program code (a3) for inputting coded multiplexed sound and moving picture sequence streams. 
The computer readable program code (b) may have computer readable program code (b3) for demultiplexing the coded 
multiplexed sound and moving picture sequence streams inputted by the computer readable program code (aS) into 
one or more transport stream packets having a coded moving picture sequence signal having a number of real inputting 
bits as the one or more first data strings. Also, the computer readable program code (f) may have computer readable 
program code (f3) for outputting coded multiplexed sound and moving picture sequence streams having a coded moving 
picture sequence signal having a number of real outputting bits less than the number of real inputting bits of the coded 
moving picture sequence signal. 

[0135] Alternatively, the aforesaid computer program product may further comprises: (g) computer readable program 
code for computing a value of system clock indicative of a start time of the computer readable program code (b), on 
the basis of a first reference time information element contained in the first coded signal; and (h) computer readable 
program code for computing an initial value of the system clock for the second coded signal on the basis of the value 
of the system clock computed by the computer readable program code (g). 

[01 36] Alternatively, in the aforesaid computer program product, the computer readable program code (c) may further 
comprise: (c51) computer readable program code for decoding the one or more transport stream packets having a 
coded video signal having a number of real inputting bits to reconstruct and output video PES packets; (c52) computer 
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readable program code for decoding the video PES packets decoded by the computer readable program code (c51) 
to reconstruct and output video elementary streams having a real inputting bits, decoding time management information 
element DTS and presentation time management information element PTS of the video elementary streams, and 
PTS^DTS flags Indicative of presence of the decoding time management information element DTS and the presentation 
time management infonnation element PTS; (c53) computer readable program code for transforming the video ele- 
mentary streams decoded and outputted by the computer readable program code (c52) into video elementary streams 
having a number of real outputting bits less than the number of real inputting bits of the video elementary streams; 
(c54) computer readable program code for generating transfomned video PES packets on the basis of the transformed 
video elementary stream transformed by the computer readable program code {c53), the decoding time management 
information element DTS, the presentation time management infonnation element PTS and the PTS^DTS flags indic- 
ative of presence of the decoding time management information element DTS and the presentation time management 
infomiation element PTS decoded and outputted by the computer readable program code (c52); and (c55) computer 
readable program code for encoding the transformed video PES packets generated by the computer readable program 
code (c54) to generate a transformed transport stream packets having a number of real outputting bits less than the 
number of real inputting bits of the transport stream packets. 

[01 37] Alternatively, in the aforesaid computer program product, the computer readable program code (b) may have 
computer readable program code for demultiplexing the first coded signal into transport stream packets having a coded 
audio signal as the one or more second data strings. 

[01 38] Alternatively, in the aforesaid computer program product, the computer readable program code (b) may have 
computer readable program code (b71 ) for demultiplexing MPEG-2 transport streams inputted at a predetermined time 
interval by the computer readable program code (a) into the one or more first data strings, the one or more second 
data strings, and the one or more third data strings, each having a number of real inputting bits. The computer readable 
program code (c) may comprises; (c71 ) computer readable program code for decoding the one or more first data strings 
to reconstruct video elementary streams having a number of real inputting bits and other Information elementary 
streams, and separating the video elementary streams from the other information elementary streams; (c72) computer 
readable program code for transforming the video elementary streams reconstructed and separated by the computer 
readable program code (c71 ) to generate output video elementary streams having a number of real outputting bits less 
than the number of real inputting bits of the video elementary streams; and (c73) computer readable program code for 
generating one or more transfomned flrst data strings having a number of real outputting bits less than the number of 
real inputting bits of the one or more first data strings demultiplexed by the computer readable program code (b71) on 
the basis of the output video elementary streams reconstructed by the computer readable program code {c72) and the 
other infonnation elementary steams reconstructed and separated by the computer readable program code (c71 ). The 
computer readable program code (d) may have computer readable program code (d71) for multiplexing the one or 
more transformed first data strings generated by the computer readable program code (c73), the one or more second 
data strings demultiplexed by the computer readable program code (b71), and the one or more corrected third data 
strings corrected by the computer readable program code (e) to generate a second coded signal to be outputted at 
the predetermined time interval. 

[01 39] Altematively, in the aforesaid computer program product, the computer readable program code (c) may com- 
prise: (c81) computer readable program code for assuming that a number of real outputting bits of the one or more 
second data strings contained in the second coded signal at the predetermined time interval is equal to a number of 
real inputting bits of the one or more second data strings contained in the first coded signal at the predetermined time 
interval; {c82) computer readable program code for assuming that a number of real inputting bits of the one or more 
third data strings contained in the second coded signal at the predetemriined time interval is equal to the number of 
real Inputting bits of the one or more corrected third data strings generated by the computer readable program code 
(e) at the predetermined time interval; (c83) computer readable program code for subtracting the number of real input- 
ting bits of the one or more second data strings contained in the first coded signal at the predetemiined time interval 
and the number of real inputting bits of the one or more third data strings contained in the second coded signal at the 
predetermined time interval from a target number of outputting bits of all data strings contained in the second coded 
signal at the predetemiined time interval to generate a value A; (c84) computer readable program code for subtracting 
a total number of real outputting bits of one or more transformed first data strings generated by the computer readable 
program code (c73) prior to the predetermined time interval from a total target number of outputting bits of one or more 
transformed first data strings generated by the computer readable program code (c73) prior to the predetermined time 
interval to generate a value B; and (c85) computer readable program code for computing a target number of outputting 
bits of the transfonned first data strings generated at the predetermined time interval by the computer readable program 
code {c73) by adding the value A and the value B. 

[0140] Alternatively, In the aforesaid computer program product, the computer readable program code (c72) may 
comprise: (c91) computer readable program code for computing a target number of outputting bits of the output video 
elementary streams on the basis of outputting bits of the second coded signal able to be outputted at the predetermined 
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time interval; (c92) computer readable program code for computing a reference ratio of outputting bits to inputting bits 
on the basis of the target number of outputting bits of the output video elementary streams computed by the computer 
readable program code (c91 ) and the real Inputting bits of the video elementary streams reconstructed by the computer 
readable program code (c71 ); and (c93) computer readable program code for computing a quantization scaling factor 
required for transforming the video elementary streams to generate the output video elementary streams, on the basis 
of the reference ratio of outputting bits to inputting bits computed by the computer readable program code (c92). 
[0141] Alternatively, In the aforesaid computer program product, the computer readable program code (c91) may 
comprise: (c1 01 ) computer readable program code for computing a ratio of a total number of real outputting bits of the 
first one or more transfonned data strings generated by the computer readable program code (c73) prior to the pre- 
determined time interval to a total number of real outputting bits of the output video elementary streams prior to the 
predetermined time interval; and (c102) computer readable program code for computing a target number of outputting 
bits of the output video elementary streams at the predetermined time inten/al on the basis of the ratio computed by 
the computer readable program code (c1 01 ). The computer readable program code (c92) may have computer readable 
program code (c1 03) for computing a reference ratio of outputting bits to inputting bits on the basis of the target number 
of outputting bits of the output video elementary streams computed by the computer readable program code (c102) 
and the real inputting bits of the video elementary streams reconstructed by the computer readable program code (c71). 
[01 42] Alternatively, in the aforesaid computer program product, the computer readable program code (c) may com- 
prise: (c1 1 01 ) computer readable program code for computing a total number of real outputting bits of the output video 
elementary streams reconstructed by the computer readable program code (c72); (c1 1 02) computer readable program 
code for computing a sum of a target number of outputting bits of the output video elementary streams at the prede- 
tennlned time interval and a total number of real outputting bits of the output video elementary streams which have 
been generated until the time when video elementary streams inputted prior to the predetermined time interval into a 
video ES buffer are consumed; (c1103) computer readable program code forjudging upon whether the total number 
of real outputting bits of the output video elementary streams computed by the computer readable program code (c1 1 01 ) 
Is greater than the sum computed by the computer readable program code (c1102); and (c1104) computer readable 
program code for tenninating the computer readable program code (c) and starting the computer readable program 
code (d) for processing the one or more transfonned first data strings when it Is judged that the sum is greater than 
the total number of real outputting bits by the computer readable program code (c1103). 

[01 43] Alternatively, in the aforesaid computer program product, the computer readable program code (d) comprises: 
(d1 201 ) computer readable program code for computing a difference by subtracting a value of lastly past presentation 
time management information element PTS of the first coded signal from a value of a synchronous time information 
element PGR located in a head position of a data string of the one or more second data strings; (d1202) computer 
readable program code for computing a difference between passing time of the data string of the one or more second 
data strings In the first coded signal and passing time of the data string of the one or more second data strings in the 
second coded signal; (d1203) computer readable program code for judging upon whether the difference computed by 
the computer readable program product {d1201) is smaller than the difference computed by the computer readable 
program code (d1202); and(d1204) computer readable program code for locating the data string of the one or more 
second data strings In a rearward position of a data string of the one or more first data strings to be located in a rearward 
position of the data string of the one or more second data strings when it Is judged that the difference computed by 
the computer readable program product (d1201) is smaller than the difference computed by the computer readable 
program code (d1202) by the computer readable program product (d1203). The computer readable program code (f) 
may have computer readable program code (f1201) for outputting the second coded signal at the predetermined time 
interval. 

[01 44] Alternatively, in the aforesaid computer program product, the computer readable program code (d) may com- 
prise: (d1301) computer readable program code for computing a difference by subtracting a value of a synchronous 
time infomnation element PGR to be past subsequently in the first coded signal from a value of a presentation time 
management information element PTS of a just past data string of the one or more second data strings; (d1302) 
computer readable program code for computing a difference between passing time of the data string of the one or 
more second data strings in the first coded signal and passing time of the data string of the one or more second data 
strings in the second coded signal; (d1303) computer readable program code forjudging upon whether the difference 
computed by the computer readable program code (d1301) is smaller than the difference computed by the computer 
readable program code (d1 302); and (d1 304) computer readable program code for locating the data string of the one 
or more second data strings in a forward position of a data string of the one or more first data strings to be located in 
a f onward position of the data string of the one or more second data strings when it is judged that the difference computed 
by the computer readable program code (d1301) is smaller than the difference computed by the computer readable 
program code (d1 302) by the computer readable program code (d1 303). The computer readable program code (f) may 
have computer readable program code (f1301) for outputting the second coded signal at the predetermined time in- 
terval. 



22 



EP1 130 927 A2 



BRIEF DESCRIPTION OF THE DRAWINGS 

[01 45] The present invention and many of the advantages thereof will be better understood from the following detailed 
description when considered in connection with the accompanying drawings, wherein: 

5 

Fig. 1 is a schematic block diagram of a rate converter for describing the basic guideline of the present invention; 

Fig. 2 is a schematic blocic diagram of a rate converter, which ensures to synchronize the output video bit streams 

with the input video bit streams on the basis of PTS and DTS contained in the input MPEG-2 transport streams; 

Fig. 3 is a block diagram showing a relationship between the video TS and the non-reduction TS contained in the 
10 input MPEG-2 transport stream and the output MPEG-2 transport stream; 

Figs. 4(a) and 4(b) are sets of graphs respectively showing the transitions of utilization volume of VBV buffer for 

the input MPEG-2 video ES and output MPEG-2 video ES while the input MPEG-2 video ES is transcoded; 

Fig. 5 is a block diagram indicating the concept of the rate control method for controlling the output bit rate of video 

bit streams In the VBR (Variable Bit Rate) fomriat perfomned during the transcoding process; 
^5 Fig. 6 is a schematic block diagram of a first preferred embodiment of the apparatus for transcoding the coded 

multiplexed sound and moving picture sequence according to the present Invention; 
. Fig. 7 is a table for PAT and PMT parameters; 

Fig. 8 is a block diagram showing the positions of non-reduction TS packets In the Input MPEG-2 transport stream 

at time interval of the predetermined duration (n); 
20 Fig. 9 is a diagram showing the transition of the number of TS packets outputted attime interval of the predetermined 

duration; 

Fig. 1 0 is a block diagram showing constructions of the video PE, the video PES and the video TS; 

Fig. 1 1 is a schematic block diagram of a MPEG-2 TS transcoder used to describe the function and construction 

of a video transcoding unit; 

25 Figs. 1 2(a) and 1 2(b) are a set of graphs respectively showing the transitions of the T-STD video elementary buffer 

utilization volume for input and output video ES; 

Figs. 1 3(a) and 1 3(b) are a set of graphs respectively showing the transition of the utilization volume of the T-STD 
audio buffer for input and output audio ES; 

Figs. 1 4(a) and 1 4(b) are a set of graphs respectively showing the transition of the T-STD video elementary buffer 
30 utilization volume for input and output video ES; 

Figs. 1 5(a) and 1 5(b) are a set of graphs respectively showing the transition of the utilization volume of the T-STD 
audio buffer for input and output audio ES; 

Fig. 16 is a diagram showing input and output MPEG-2 TS which may cause the buffer overflow; 
Fig. 1 7is a set of graphs showing the transition of the utilization volume of the T-STD video buffer and the T-STD 
55 audio buffer; 

Fig. 18 is a schematic block diagram showing a conventional transcoder; 

Fig. 19 is a flowchart showing the flow of the rate control operation of MPEG-2 perfonned by the conventional 
transcoder shown in Fig. 18; 

Fig. 20 is a schematic block diagram showing a conventional transcoder; 

Fig. 21 is a flowchart showing the flow of the rate control operation of MPEG-2 performed by the conventional 
transcoder shown in Fig. 20; 

Fig. 22 is a schematic block diagram showing a conventional transcoder; 

Fig. 23 is a flowchart showing the flow of the rate control operation of MPEG-2 performed by the conventional 
transcoder shown in Fig. 22; 
^5 Fig. 24 is a diagram showing renderings of an environment in which the present invention is utilized; 

Fig. 25 is a schematic block diagram showing an apparatus of the simple combination of the MPEG-2 transport 

stream decoder and the MPEG-2 transport stream encoder; and 

Fig. 26 is a schematic block diagram showing a conventional transcoder. 

50 DESCRIPTION OF THE PREFERRED EMBODIMENTS 

[01 46J Referring now to Fig. 1 of the drawings, the basic concept of the present Invention will be described herelnlater. 
[01 47] The rate converter 600 is shown in Fig. 1 as comprising a M PEG-2 transport stream demultiplexer 61 0, herein- 
later refen^ed to as "MPEG-2 TS demultiplexer 610", a MPEG-2 transport stream multiplexer 620, hereinlater referred 
55 to as "MPEG-2 TS multiplexer", a MPEG-2 video transcoder 640, and a system controller 650. 

[01 48] The MPEG-2 TS demultiplexer 61 0 is operated to input MPEG-2 transport stream and demultiplex the inputted 
MPEG-2 transport streams into video transport streams, hereinlater refenred to as "video TS", audio transport streams, 
hereinlater refen-ed to as "audio TS", and system information transport streams, hereinlater referred to as "system 
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information TS", and output the video TS, the audio TS, and the system infomnation TS. 

[0149] From the foregoing description, it will be understood that the MPEG-2 TS demultiplexer 610 comprises the 
inputting means and the multiplexing means to perform two functions consisting of inputting operation and demulti- 
plexing operation. The video TS, the audio TS, and the system Information TS constitute one or more first data strings, 
one or more second data strings, and one or more third data strings, respectively 

[0150] The MPEG-2 TS multiplexer 620 Is operated to multiplex the video TS, the audio TS, and the system infor- 
mation TS Into MPEG-2 transport streams, and output the thus multiplexed MPEG-2 transport streams The MPEG-2 
TS multiplexer 620 constitutes the multiplexing means. 

[0151] The MPEG-2 video transcoder 640 is operated to input the video TS, transcode the video TS, and output the 
video TS having the number of bits less than that of the inputted video TS. The M PEG-2 video transcoder 640 constitutes 
the transfomning means. 

[0152] The system controller 650 is operated to input the system information TS, modify or not modify part of fixed 
codes of the system infomnation TS in accordance with the output MPEG-2 transport streams, and output the partly 
modified or not modified system information TS. The system controller 650 constitutes the connecting means. 
[01 53] The rate converter 600 is operated to compress only the video bit streams, which have extremely large infor- 
mation volume. The compression operation is carried out through the steps of (a) operating the MPEG-2 TS demulti- 
plexer 610 to demultiplex the Input MPEG-2 transport streams into video TS, audio TS, and system information TS; 
(b) operating the MPEG-2 video transcoder 640 to transcode the video TS to generate output video TS having infor- 
mation volume less than the video TS; (c) operating the system controller 650 to modify or not modify part of fixed 
codes of the system infomnation TS in accordance with the output MPEG-2 transport streams; (d) operating the MPEG- 
2 TS multiplexer to multiplex the video TS thus transcoded, the audio TS, and the eventually partly-modified or not 
modified system infonmation TS into output MPEG-2 transport streams, and output the output MPEG-2 transport 
streams. As will be understood from the description, the system controller 650 Is operated, instead of the system 
Infomnation decoder 91 7 and the system infomnation encoder 937 of the apparatus 900, to modify or not modify part 
of fixed codes of the system infomnation bit streams in response to the output M PEG-2 transport streams, and eventually 
replace the thus partly-modified system Information bit streams with the original system Information bit streams. The 
rate converter 600 is simplified in comparison with the conventional art such as the apparatus 900. 
[0154] As will be understood from the foregoing descriptions, the rate converter 600 ensures that the following prob- 
lems (1 ) to (3) are solved with respect to the apparatus 900 of the simple combination of the MPEG-2 TS decoder and 
the MPEG-2 TS encoder. 

(1) Amount of process is Increased. 

(2) Quality of pictures Is deteriorated. 

(3) Frame realignment causes delay. 

[0155] Fig. 2 shows a rate converter 700, which ensures to synchronize the output video bit streams with the Input 
video bit streams on the basis of PTS (Presentation Time Stamp: presentation time management information element) 
and DTS (Decoding Time Stamp: decoding time management information element) contained in the Input MPEG-2 
transport streams. 

[0156] The rate converter 700 is shown in Fig. 2 as comprising a MPEG-2 TS demultiplexer 710, a MPEG-2 TS 
multiplexer 720, a video TS decoder 741 , a video PES decoder 742, a video ES transcoder 744, a video PES packet 
generator 745, a video TS packet generator 746. 

[0157] The MPEG-2 TS demultiplexer 710 Is operated to demultiplex the Inputted MPEG-2 transport streams Into 
video TS and other TS and to output the video TS and the other TS. 

[0158] The MPEG-2 TS multiplexer 720 is operated to input the video TS and the other TS and to output the output 
MPPEG-2 TS transport streams. 

[0159] The video TS decoder 741 is operated to input the video TS, to decode the video TS into video PES, and to 
output the video PES. The video TS decoder 741 constitutes the video transport stream packet decoding unit. 
[0160] The video PES decoder 742 is operated to Input the video PES, to decode the video PES Into the PTS, the 
DTS, other information element, and video ES (ES: elementary stream), and to output the PTS, the DTS and the video 
ES. The video PES decoder 742 constitutes the video PES packet decoding unit. 

[01 61 ] The video ES transcoder 744 Is operated to Input the video ES, to transcode the video ES to generate trans- 
coded video ES having the number of bits less than the number of bits of the video ES. The video ES transcoder 744 
constitutes the transforming unit. 

[0162] The video PES packet generator 745 Is operated to input the PTS, DTS, the other infonmation element and 
the transcoded video ES to generate and output the output video PES. The video PES packet generator 745 constitutes 
the video PES packet generating unit. 

[0163] The video TS packet generator 746 Is operated to Input the output video PES to generate and output the 
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output video TS having the number of bits less than that of the inputted video TS. The video TS packet generator 746 

constitutes the video transport stream generating unit. 

[01 64] The rate converter 700 is operable to ensure the synchronization of the output video bit streams with the input 
video bit streams through the steps of: (a) decoding the video PES into the video ES, the corresponding PIS, the 

5 corresponding band other information element; (b) temporally storing the PTS and the DTS; and (c) generating the 
output video PES from the transcoded video ES, the corresponding PTS, the corresponding DTS, and the other infor- 
mation element so that the PTSs and the DTSs In the Input video elementary streams of the video bit streams contained 
in the input MPEG-2 transport streams are matched with those in the corresponding video elementary streams of the 
output video bit streams contained in the output MPEG-2 transport streams as well as the PTSs and the DTSs in the 

10 audio frames of the audio bit streams contained in the input MPEG-2 transport streams are matched with those In the 
corresponding audio frames of the audio bit streams contained in the output MPEG-2 transport streams. 
[0165] Referring to Fig. 3 of the drawings, there is shown a relationship between video TS and non-reduction TS 
contained in the input MPEG-2 transport stream and the output MPEG-2 transport stream. In Fig. 3. the bit rate of the 
output MPEG-2 transport stream is reduced to the half of that of the input MPEG-2 transport stream. 

IS [01 66] Here, "non-reduction TS packets" are ail the TS packets excluding two types of packets consisting of transport 
packets including video bit streams contained in the input MPEG-2 transport stream, and TS packets apt to change in 
accordance with the control state in the system such as PAT and PMT. 

[0167] As will be understood from Fig. 3, the non-reduction TS packets are interposed between the packets to be 
reduced, hereinlater referred to as "reduction-object packets", in accordance with a ratio of the output bit rate to the 
20 input bit rate, thereby making it possible that the non-reduction TS packets contained in the output MPEG-2 transport 
streams arrive at the decoder at the same time at which the non-reduction TS packets contained in the input MPEG- 
2 transport streams are supposed to arrive at. 

[0168] Referring to Fig. 4 of the drawings, the transition of utilization volume of a Video Buffering Verifier buffer, 
hereinlater refen^ed to as "VBV buffer" (Video Buffering Verifier: a parameter indicative of the size of the virtual buffer 

25 used for controlling the number of generating bits) and DTS (Decoding Time Stamp: decoding time management in- 
formation) while the bit rate of video bit streams contained in the output MPEG-2 transport streams is reduced to the 
half of that of video bit streams contained in the input MPEG-2 transport streams In the case of (a) and the case of (b). 
[0169] In the case of (a), the number of bits of l-pictures are not reduced while the number of bits of P-pictures and 
B-plctures are reduced. In the case of (b), on the other hand, the bit numbers of l-plctures, P-pictures, and B-plctures 

30 are evenly reduced. This means that the ratio of l-pictures, P-pictures, and B-pictures in the Input video bit streams 
remains the same as that in the output video bit streams. This means that the ratio of the number of bits constituting 
a frame In the output bit streams to the number of bits constituting the same frame In the output bit streams is equal 
to the ratio of the output bit rate to the input bit rate. 

[0170] In Fig. 4, the upper graph shows the transition of utilization volume of the VBV buffer for the video elementary 
35 streams contained In the input bit streams before the transcoding operation, and the lower graph shows the transition 
of utilization volume of the VBV buffer for the video elementary streams contained in the output bit streams after the 
transcoding operation. "B" indicates the size of a receiving buffer, "B(n)*" indicates the VBV buffer utilization volume 
just before the n-th picture is decoded, and "B(n)" indicates the VBV buffer utilization volume just after the n-th picture 
is decoded and the number of bits for the size of the picture frame is removed from the buffer. The VBV buffer utilization 
40 volume must fluctuate within a range of 0 and B. The slope of a line segment indicates a bit rate. 

[01 71 ] The receiving buffer waits until the time indicated by DTS to start decoding a picture frame consisting of input 
video elementary streams while the receiving buffer waits until the time indicated by DTS' to start decoding the picture 
frame consisting of the output video elementary streams. 

[0172] In the case of (a), DTS' is greater than DTS. As will be understood in Fig. 4(a), a delay of (DTS' - DTS) occurs. 

45 The difference of (DTS' - DTS) causes not only the delay in the output MPEG-2 transport streams, but also disables 
the synchronization with the audio bit streams. On the other hand, if the DTS is attached to the output bit streams as 
an information element indicative of the starting time of the decoding operation, some of the video elementary streams 
constituting one picture frame may not arrive at the MPEG-2 transport stream decoder before the time indicated by 
the DTS, thus, the MPEG-2 transport stream decoder must wait until all of the video elementary streams constituting 

50 the picture frame arrive so as to decode the whole picture frame. 

[0173] In the case of (b), DTS' is equal to DTS. This means that all of the video elementary streams constituting one 
picture frame will arrive at the MPEG-2 transport stream decoder until the time indicated by the DTS, thereby making 
it possible to start decoding the picture frame at the time indicated by the DTS. 

[0174] As will be understood from the foregoing descriptions, the rate control method must satisfy the condition of 
55 DTS' =DTS, that is to say, the ratio of the number of bits constituting one frame in the output bit streams to the number 
of bits constituting the same frame in the input bit streams must be equal to the ratio of the output bit rate to the input 
bit rate. 

[0175] Fig. 5 Is a block diagram Indicating the concept of the rate control method for controlling the output bit rate 



25 



EP1 130 927 A2 



of video bit streams in tlie VBR (Variable Bit Rate) format performed during the transcoding process. In Rg. 5, the input 
IVIPEG-2 transport streanns at an input bit rate is converted Into the output IVIPEG-2 transport streams at the target bit 
rate, which is the half of the input bit rate. 

[01 76] Each of the MPEG-2 transport stream consists of packets of 1 88 bytes. The number of packets to be inputted 
5 at time interval of a predetermined duration is always the same. This means that the number of packets inputted at 
time interval of a predetemiined duration can be computed on the basis of the product of the input bit rate and the 
predetermined time duration . This leads to the fact that a target number of packets to be outputted at time interval of 
the predetermined duration can be computed in the similar manner of computing the number of packets to be inputted 
at time Interval of the predetermined duration. 
10 [0177] Furthemriore, the number of video TS and the number of non-reduction TS contained in the input MPEG-2 
transport streams can be computed on the basis of infonnation elements acquired when the Input MPEG-2 transport 
steams are demultiplexed. Provided that the non-reduction transport streams are not to be compressed, the target 
number of output video TS packets at time interval of a predetemnined duration (n) can be computed by subtraction 
of the number of packets of the non-reduction TS and the number of PAT and PMT packets to be outputted at time 
'5 interval of the predetemiined duration (n) from the target number of the total packets to be outputted at time interval 
of the predetemiined duration (n). 

[0178] As will be understood from the foregoing descriptions, the rate control for controlling the output bit rate of 
video bit streams in the VBR fonnat can be performed basically by transcoding the video elementary streams decoded 
from the input video TS to reconstruct the output video elementary streams so as to output the target number of the 

20 output video TS packets. 

[01 79] There is another problem which makes it difficult to detemriine the target number of the output video TS pack- 
ets. While transcoding the MPEG-2 transport streams, the video elementary streams are converted to PES (PES: 
Packetized Elementary Stream) packets and TS (TS: Transport Stream) packets, and as a consequence, new packet 
headers are generated and attached to the PES packets andTS packets, thereby causing overhead. The exact number 

25 of packet headers thus newly generated, however, cannot be estimated beforehand. The target bit number of the output 
video elementary streams to be outputted at time inten/al of the predetermined duration (n), therefore, is determined 
through the steps of (1) computing the overhead generated and accumulated through the process of converting the 
video elementary streams to PES packets and TS packets from the time interval of the predetemiined duration (0) to 
the time interval of the predetemiined duration (n-1); and (2) computing the target bit number of the output video 

30 elementary streams at time interval of the predetermined duration (n) in consideration of the overhead generated and 
accumulated from the time interval of the predetemiined duration (0) to the time inten/al of the predetermined duration 
(n-1) computed in the step (1). 

[0180] The basic guideline of the MPEG-2 TS transcoder algorithm for controlling the target bit rate based on the 
method according to the present invention will be described hereinlater. 

35 

Basic guideline: 

[01 81 ] The number of bits contained in the input MPEG-2 transport stream must be reduced in order to control the 
output bit rate. 

40 [0182] The MPEG-2 transport stream, hereinlater refen^ed to as "MPEG-2 TS", comprises a system control signal 
and a code signal. The code signal includes the video bit stream and the audio bit stream. The system control signal 
is constituted by a PAT (Program Association Table), a PMT (Program Map Table), PID (Packet Identification), CAT 
(Conditional Access Table), and NIT (Network Information Table). The PAT is part of the service Information. For each 
program accessible on the current transport stream, a PAT gives the PID of its PMT, which will be described hereinlater. 
45 The PMT is part of the service Infomiation. A PMT gives Information about a program within the current transport 
stream and makes the mapping between program components and elementary streams. PMTs contain a list of ele- 
mentary streams that may be identified by a stream type and a track value. A PMT also contains the Program Clock 
Reference (PCR) PID. The PID is an integer used by the kernel to uniquely Identify a process or a packet. The CAT is 
part of the service information. A CAT provides conditional access systems infonnation and the Entitlement Manage- 
so ment Messages (EMM) stream PID and description. Each CAT contains one loop of descriptors. The NIT is part of the 
service information. An NIT contains information about the network and the physical medium 
[0183] As will be understood, the system control signal Is directly related to the service. The reduction of the system 
control signal will result in the failure or degradation of the actual service. This means that the system control system 
cannot be reduced. Accordingly, the video bit stream or the audio bit streams are considered to be reduced. 
55 [0184] MP@ML (a "confomnancepoinfthattheMPEGcommitteedetermined to adoptforconsumer quality standard) 
indicates that the size of the video bit stream is In a range of 4 Mbps to 15 Mbps, and the size of the audio bit stream 
is less than 384 kbps. Because of the fact that the video bit stream has the extremely large number of bit rate, the 
basic guideline of the MPEG-2 transcoder algorithm is targeted for compressing only the video bit stream, which is 
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regarded to occupy extremely large information volume in the MPEG-2 transport stream. 

[0185] The predetermined duration (n) is computed by dividing the total number of bits of the MPEG-2 TS packets 
Inputted to the MPEG-2 TS transcoder by the Input MPEG-2 transport stream bit rate, herelnlater refen-ed to as, 
"TSBJn" [bps]. The time interval of the predetemnlned duration (n) expires at the time when the total number of the 
input MPEG-2 TS pacl<ets, hereinlater referred to as, "AILTSJn" satisfies the equation (32) as follows: 



TSBi„xbf_time 

A11TSi„ = X n equation (32) 



10 



where "188" is a packet length [byte/packet], and "8" is the number of bits per byte [bit/byte]. 
[01881 The transcoder, however, does not immediately start transcoding the inputted MPEG-2 transport streams if 
already started MPEG-2 transport streams are inputted, for instance, continuous bit streams such as bit streams car- 
rying the continuous digital broadcast data elements are inputted. In the case, the transcoder firstly detects the first 
IS PAT and PMT In the Inputted bit streams, sets the time when the first PAT and PMT are detected as the initial time of 
the predetermined duration. 

[01 87] Referring to Fig. 6 of the drawings, there Is shown a schematic block diagram of a first prefen-ed embodiment 
of the transcoder for transcoding the coded multiplexed sound and moving picture sequence according to the present 
invention. 

20 [0188] The transcoder 200 is as shown in Fig, 6 as comprising an Input MPEG-2 TS demultiplexer 21 0, an output 
MPEG-2 TS multiplexer 220. a non-reduction TS buffer 230, a PAT, PMT generator 280, and a video TS processor 
240. The video TS processor 240 comprises a video TS packet decoder 241 , a video PES packet decoder 242, a video 
PES packet generator 245, a video TS packet generator 246, a video TS packet buffer 247. and a video transcoding 
unit 1 00. The video transcoding unit 1 00 has a video ES buffer 243 and a video ES transcoder 244. 

25 [01 89] The input MPEG-2 TS demultiplexer 21 0 constitutes the inputting means and the demultiplexing means, the 
output MPEG-2 TS multiplexer 220 constitutes the multiplexing means, the PAT, PMT generator 260 constitutes the 
correcting means, and the video transcoding unit 1 00 constitutes the transforming means. Every constitutional element 
is operated in parallel and synchronization with one another to perform the real time operation. 
[01 90] The operation of the video transcoding unit 1 00 will be described hereinlater 

30 [0191] The input MPEG-2 TS demultiplexer 210 Is adapted to identify a first reference time information element, 1. 
e., PGR (Program Clock Reference), contained In the Input MPEG-2 transport stream to compute the initial value of 
the system clock, i.e., PCR__offset, indicative of the start time of the demultiplexing operation on the basis of PGR, 
and output PGR_offset to the output MPEG-2 TS multiplexer 220. The output MPEG-2 TS multiplexer 220 is operated 
to compute the initial value of the system clock used for the output MPEG-2 transport stream on the basis of 

35 PCR^otfset. 

[01 92] As will be understood from the foregoing descriptions, the input MPEG-2 TS demultiplexer 21 0 and the output 
MPEG-2 TS multiplexer 220 Is designed to perfomn the function of the reference time setting means, and PGR senses 
as the value of system clock indicative of the start time of the demultiplexing means. 

[01 93] The input MPEG-2 transport stream consists of TS packets. There are three types of the TS packets such as 

40 the PAT, PMT packets, the non-reduction TS packet, and the video TS packets 

[0194] The TS packet of the same type has the similar payload. The Input MPEG-2 TS demultiplexer 210 Is adapted 
to input the input MPEG-2 transport stream consisting of TS packets, Identifying PIDs of TS packet headers, and 
demultiplex the TS packets into the PAT, PMT packets, the non-reduction TS packets, and the video TS packets at 
time interval of the predetermined duration (n) on the basis of the PIDs. The video TS packets, the non-reduction TS 

45 packet, and the PAT, PMT packets sen/e as one or more first data strings, one or more second data strings, and one 
or more third data strings, respectively. 

[0195] The Input MPEG-2 TS demultiplexer 210 Is adapted to identify the type of the TS packets on the basis of the 
PIDs and the payloads of the TS packets, and carry out three different operations according to the type of the TS 
packets as follows. 

50 

(1) PAT, PMT packets 

The input MPEG-2 TS demultiplexer 21 0 Is adapted to decode the PAT, PMT packets, extract PAT, PMT data 
information elements (see Fig. 7) from the PAT, PMT packets, and output the PAT, PMT data infomriation elements 
thus extracted to the PAT, PMT generator 260. The PAT, PMT packets may be modified or not modified in accord- 
55 ance with the transmitting path which the output MPEG-2 transport stream flows at the target bit rate. 

(2) Non-reduction TS packets such as the audio TS packets, the system control TS packets excluding the above 
PAT, PMT packets, and null packets (Invalid packets) 

The input MPEG-2 TS demultiplexer 21 0 is adapted to store the positions of the non-reduction TS packets at 
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time interval of the predetermined duration (n), and output the non-reduction TS packets to the non-reduction TS 
buffer 230. The output MPEG-2 TS multiplexer 220 is adapted to read the stored positions of the non-reduction 
TS packets while multiplexing TS packets of the three types Into the output MPEG-2 transport stream so that the 
non-reduction TS packets contained In the output MPEG-2 transport stream arrive at the time at which the non- 

s reduction TS packets contained in the input MPEG-2 transport stream are supposed to arrive at. 

Fig. 8 shows positions of the non-reduction TS packets In the Input MPEG-2 transport stream at time Interval 
of the predetemnlned duration (n). NonV_Run(i) indicates the position of the l-th non-reduction TS packet, i.e., 
the number of TS packets Interposed between the first TS packet and the i-th non-reduction TS packet at time 
Interval of the duration (n). 

10 (3) Video TS packets 

The input MPEG-2 TS demultiplexer 210 Is adapted to delete the video TS packets having no payload, and 
output the video TS packets to the video TS packet decoder 241 . 

[0196] The input MPEG-2 TS demultiplexer 210 is adapted to send a signal indicative of the end of the time interval 
15 of the duration (n), to the non-reduction TS buffer 230 and the video TS packet buffer 247 at the time when the number 
of the input MPEG-2 TS packets satisfies the aforesaid equation (32). 

[0197] Furthermore, the Input MPEG-2 TS demultiplexer 210 is adapted to decode the PCR^PID TS packet Indicated 
by PMT In the input MPEG-2 transport stream to reconstruct the PCR at time interval for the predetermined duration 
(0), compute the total byte number of the input MPEG-2 transport stream inputted to the Input MPEG-2 TS demultiplexer 
20 210 prior to the PCR, and compute the PCR^offset of the system clock , I.e., the start time of the demultiplexing 
operation at which the first byte is Inputted to the Input MPEG-2 TS demultiplexer 210 on the basis of the bit rate of 
the input MPEG-2 transport stream and the total byte number of the input MPEG-2 transport stream Inputted prior to 
the PCR In accordance with the equation (33) as follows: 



25 



r.^r. r,^« 27000000 X (( TS„,-1 ) X 1 88 X 8+P CR^>ointX 8) 

PCR^ffser^irstPCR' — equation (33) 



where "f Irst^PCR" is the first PCR, TS_cnt is the total number of the Input MPEG-2 TS packets Inputted prior 
30 to the f irst_PCR, including the TS packet which contained the encoded f irst_PCR, PCR jaoint is the total byte number 
from the head byte to the byte which contains the end bit of program_clock_reference_base in the TS packet which 
contains the encoded f lrst_PCR, TSB Jn Is the bit rate of the input MPEG-2 transport stream, and "27000000" is the 
MPEG-2 reference clock, which is 27MHz. 

[0198] The PAT, PMT generator 260 is operated to generate PAT, PMT packets for the output MPEG-2 TS on the 
35 basis of the PAT, PMT data Infomnation elements (see Fig. 7) received from the input MPEG-2 TS demultiplexer 21 0 
when the total number of the output TS packets outputtedattime interval of the predetermined duration (n),l.e.,TS_out, 
satisfies the equation (34) as follows; 



40 ij— 1 

188 X8X (TS„„^-f-2 A7iT5,„,(/ ))>freq X TSS,„,X 

...equation (34) 

45 

where all_ TS_out(i) Is the number of the output TS packets at time interval for the predetemiined duration (1), 
S_out is the total number of times PAT, PMT has been outputted until the time when the total number of the output TS 
packets satisfies the equation (34), and TSB_out[bps] Is the bit rate of the output MPEG-2 transport stream, freq is a 
time interval that PAT, PMT is transmitted. The transcoder 200 is designed to set PCR to PMT. It is stipulated that PCR 
50 should be transmitted at time intervals of 0.1 second or less. As a result, freq shall be 0.1 second or less, i.e., freq ^ 0.1 . 
[0199] The initial value of PCR is PCR^offset, which Is computed by the input MPEG-2 TS demultiplexer 210 in 
accordance with the equation (35). 



55 
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27000000 X((rs,,,+2 AllTS,,, (/)) X 188+?Ci?.po/flt )X 8 
PCi?= — i^O 

+ P^^oflset . . .equation (35) 



[0200] The output MPEG-2 TS multiplexer 220 is operated to start Inputting the TS packets at time Interval of the 
10 predetermined duration (n) upon receiving a signal indicative that non-reduction TS buffer 230 and video TS paclcet 
buffer 247 are filled up. Here, the number of video TS packets inputted at time interval of the predetermined duration 
(n) will be referred to as "Video_TS_out(n)", the number of non-reduction TS packets inputted at time Interval of the 
predetermined duration (n) will be referred to as "NonV_TS_out(n)", and the number of the PAT, PIVIT packets to be 
outputted at time interval of the predetermined duration (n) will be refen-ed to as "PATPMT_out(n)", hereinlater. 
IS PATPMT_out(n) is determined to be 2 If the equation (36) Is satisfied, otherwise 0. The equation (36) is as follows: 



188X8 X i; tAUTS,,t{i )>freq X TSB,^,X S,,, 

20 '"O 

.•.equation (36) 

where taILTS_out(i) Is the target number of TS packets to be outputted at time interval of the predetemiined 
duration (i). 

25 [0201] The number of the video TS packets contained in the buffer at time interval of the predetermined duration (n), 
hereinlater referred to as "Vldeo_TS_buff(n)", is computed in accordance with the equation (37), and the number of 
the non-reduction TS packets contained In the buffer at time interval of the predetermined duration (n), hereinlater 
referred to as "NonV_TS_butf(n)", is computed in accordance with the equation (38) as follows: 

30 

Video_TS_buff(n)=Vldeo_TS_out(n) +diff_TS_out(n) equation (37) 



NonV_TS_buff(n) =NonV_TS_out(n) + PATPMT_out(n) equation (38) 

35 

where diff_TS_out(n) will be described hereinlater. 
[0202] The output MPEG-2 TS multiplexer 220 is designed to control input and output synchronization on the basis 
of the synchronization Infomnation element, I.e., NonV_Run(i) Inputted from the Input MPEG-2 TS demultiplexer 210. 
The output MPEG-2 TS demultiplexing process is carried out through the three following steps. Step 1 
40 [0203] Compute the target number of the TS packets to be outputted at time interval of the predetermined duration 
(n), i.e., target_TS_out, in consideration of the difference between the total of the target numbers of the TS packets 
to be outputted at time interval from the 0-th, I.e., duration (0) to the (n)-th, i.e., duration (n) and the total number of 
TS packets outputted at time interval from 0-th duration (0) to the (n-l)-th, i.e., duration {n-1) In accordance with the 
equation (39) as follows: 

45 

50 ...equation (39) 



where alLTS_out(i) is the number of the TS packets outputted at time Interval of the l-th duration (I). Step 2 
[0204] Judge upon whether the total number of the output TS packets outputted at time inten/al of the predetennined 
duration (n), i.e., TS_out, satisfies the equation (34). PAT, PMT is generated and outputted by the PAT, PMT generator 
260, and T S__out and S_out are Incremented by two if it is judged that the total number of the output TS packets 
outputted at time Interval of the predetermined duration (n), I.e., TS_out, satisfies the equation (34). Step 3 
[0205] Output one video TS packet if the below-stated equation (40) is satisfied. Othenvise, one non-reduction TS 
packet is outputted and i is Incremented by one. i is the number of the non-reduction TS packets to be outputted at 



29 



EP1 130 927 A2 

time interval of the predetermmed duration. The equation (40) is as follows: 



A/on V Run{i)xTSB^^ 
"^^out^ =i=cQ equation (40) 

where the nunnber of the outputted TS paclcets Is added to TS_out whenever the TS packets are outputted. Step 4 
[0206] The output MPEG-2 TS multiplexing operation is temriinated if a termination condition, which will be described 
below is satisfied. Othenwise, go back to the step 2 and repeat the step 2 and step 3 until the termination condition is 
10 satisfied. 

[0207] The temrilnation condition Is as follows: 



15 



If \/ldeo_TS_buff(n)+NonV_TS_buff(n) is greater than targetjrS_out. TS_out=target_TS_out 

Otherwise, 



TS_out=Video_TS_buff(n)+NonV_TS.buff(n) 

20 

[0208] If Vldeo_TS.buff(n)+NonV_TS_buff(n) is greater than target_TS_out, compute dltfVTS„out(n-i.1) in ac- 
cordance with the equation (41) after the output MPEG-2 TS multiplexing operation is temriinated: 

25 diffVTS_out(n+1 ) Vldeo_TS_buff(n) +NonV_TS_buff(n)— target_TS_out equation (41) 

where dlffVTS_out(n+1) is the difference between the number of TS packets actually outputted to the output 
MPEG-2 TS multiplexer 220 at time interval of the duration (n) and the target number of TS packets to be outputted 
the output MPEG-2 TS multiplexer 220 at time Interval of the duration (n). In other words, dlffVTS_out(n+1) is the 

30 difference between the number of video TS packets actually outputted at time interval of the duration (n) and the target 
number of video TS packets to be outputted at time interval of the duration (n). This means that all of the TS packets 
to be outputted at time Interval of the predetenmlned duration (n + 1 ) for the mount of dlffVTS_out(n+1 ) are video TS 
packets. The output MPEG-2 TS multiplexer 220 is operated to perform stuffing operation, i.e., output null packets 
(invalid packets) to the output MPEG-2 TS so that the output bit rate of the output MPEG-2 TS comes closer to the 

35 target bit rate if the following equation (42) is satisfied: 

i; tAUTS,,t{i ) - ( TS.^t +"2 ^^JTS,^t a )) > STUFF_TH 

40 i=0 i=0 

...equation (42) 

where STUFF_TH is a threshold value. If the number of the actually outputted packets is smaller than the target 
number of the TS packets to be outputted by more than STUFF_TH, the stuffing process is performed. 

45 [0209] Fig. 9 shows transitions of the target number of TS packets to be outputted at time interval of the predetermined 
duration, i.e., target_TS__out with respect to diff V_TS_Qut(n). The video TS packet buffer 247 is adapted to store the 
video TS transcoded by the video transcoding unit 100. The non-reduction TS buffer 230 is adapted to store the non- 
reduction TS packets demultiplexed by the Input MPEG-2 TS demultiplexer 210, and output the stored non-reduction 
TS packets to the output MPEG-2 TS multiplexer 220 in synchronous with the video TS packet buffer 247 in the video 

50 TS processor 240. 

[021 0] The video TS processor240 is adapted to decode the video TS inputted by the input MPEG-2 TS demultiplexer 
21 0 to reconstruct the video ES, transcode the video ES to generate the output video ES, which has infonnation volume 
less than the video ES, to generate the output video TS, and output the output video TS thus generated to the output 
MPEG-2 TS multiplexer 220. 
55 [021 1 ] The constitutional elements of the video TS processor 240 will be described hereinlater 

[0212] The video TS packet decoder 241 Is designed to decode the video TS inputted by the input MPEG-2 demul- 
tiplexer 210 to generate the video PES, and output the video PES thus generated to the video PES packet decoder 
242. The video TS packet decoder 241 constitutes the video transport stream packet decoding unit. 
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[021 3] The video PES packet decoder 242 is designed to decode the video PES received from the video TS packet 
decoder 241 to generate the video ES, and output the video ES thus generated to the video ES buffer 243. The video 
PES packet decoder 242 is adapted to detect a first sequence header upon starting the transcoding operation, and 
delete bit streams prior to the first sequence header if there are provided bit streams prior to the first sequence header. 
[021 4] The video PES packet decoder 242 is designed to output the video ES to the video ES buffer 243 in response 
upon receiving a signal indicative that the video ES buffer 243 is empty from the video transcoding unit 100 at time 
interval of the predetennlned duration (n). 

[0215] Furthermore, the video PES packet decoder 242 Is designed to output PTS (I), DTS (i) and PTS_DTS_flag 
(I), which are synchronous infomnation elements obtained at the time when the video PES is decoded, to the video 
PES packet generator 245. Here. I is a picture number starting from the first picture inputted at the time when the 
transcoding operation started. Thus decoded synchronous infomnation elements, I.e., PTS (I), DTS (i) and 
PTS_DTS_flag (I) make it possible to match the synchronization infomriation elements of pictures in the input MPEG- 
2 transport streams with those of the corresponding pictures in the output IVIPEG-2 transport streams, respectively. 
The video PES packet decoder 242 constitutes the video PES packet decoding unit. 

[021 6] The operation of the video PES packet generator 245 and the video TS packet generator 246 will be described 
hereinlater. 

[021 7] The video PES packet generator 245 is adapted to input and encode the PTS (I), DTS (I) and PTS_DTS Jlag 
(i) and the video ES to generate the output video PES. The video PES packet generator 245 constitutes the video PES 
packet generating unit. 

[0218] The video TS packet generator 246 is adapted to input the video PES and output the output video TS. The 
video TS packet generator 246 constitutes the video transport stream packet generating unit. 
[021 9] Whenever video ES constituting one TS is generated by the video transcoding unit 1 00, the video PES packet 
generator 245 and the video TS packet generator 246 are collectively adapted to packetize the video ES to generate 
the video TS and store the video TS thus generated in the video TS packet buffer 247. Fig. 1 0 shows constructions of 
the video ES packet, the video PES packet and the video TS packet. As shown in Fig. 1 0, each of PES packets consists 
of one picture information element of the video ES packet. 

[0220] The PTS (I), DTS (i) and PTS_DTSJIag (i) inputted from the video PES packet decoder 242 are attached to 
the PES header of the con^esponding picture I as the synchronization infonnation elements PTS, DTS, and 
PTS_DTS_flag so as to ensure that the bit streams which have been transcoded and thus compressed will be syn- 
chronized with the bit streams which have not been transcoded nor compressed. 

[0221] The video TS packet decoder 241 constitutes the video elementary stream decoding unit, the video PES 
packet decoder 242, the video transcoding unit 1 00 and the video PES packet generator 245 collectively constitutes 
the video elementary stream transfonming unit, and the video TS packet generator 246 constitutes the transformed 
first data srring generating unit. 

[0222] The synchronous information elements, i.e., PTS and DTS are, however, time stamps to be firstly accessed 
in the PES packet. If a PES packet having two or more pictures of video frame is inputted, the second or later pictures 
of the video frame in the PES packet cannot have their own PTS and DTS. 

[0223] The ARIB specification stipulates that one video PES should consist of video data elements of one frame. 
According to the ARIB specification, the PTS (i) and DTS (I) must be generated and attached to the aforesaid second 
or later pictures of the video frame in the PES packet. The PTS (I) and DTS (I) are computed in accordance with the 
equations as follows: 

PTS(i)=PTS(i-1 )+90000/FrameRate 



DTS(i)=DTS(i-1 )+90000/FrameRate 

[0224] Provided that an Inputted non-ARIB conformable MPEG-2 transport stream does not have to be outputted in 

the ARIB format, the output MPEG-2 transport stream may contain two frames in one PES. 

[0225] Come back to Fig. 6, the video transcoding unit 100 Is assumed to terminate the transcoding operation at 
time Inten/al of the predetermined duration (n), and all the video TS packets contained in the video TS packet buffer 
247 are outputted to the output MPEG-2 TS multiplexer 220 at the time when the number of bytes of the output video 
elementary stream generated at time interval of the predetermined duration (n), i.e., VES^out, satisfies the equation 
(43) as follows; 
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y^S^ut +2 VideoES^Ji ) > TH,^,{n) 



...equation (43) 



i=0 



5 



where TH_out(n) is a reference threshold value of the transcoding operation and, here, an expected value of 
the total byte number of the video elementary streams outputted from the time interval of the predetermined duration 
(0) to the time interval of the predetermined duration (n). Beside the expected value of the total byte number of the 

10 video elementary streams outputted from the time Inten/al of the predetemnined duration (0) to the time interval of the 
predetermined duration (n), other values can be considered to be the reference threshold value, TH_out(n), which will 
be described hereinlater. When the equation (43) is satisfied, VES_out is assumed to be Vldeo_ES_out(n). 
[0226] The video TS packet buffer 247 is adapted to store the video TS packets generated by the video TS packet 
generator 246, and output the video TS packets to the output MPEG-2 TS multiplexer 220 in synchronous with the 

15 non-reduction TS buffer 230, which stores the non-reduction TS packets. 

VBR video ES rate conversion control method: 

[0227] The construction of the video transcoding unit 100 in the MPEG-2 TS transcoder 200 and the VBR video 
20 stream rate conversion algorithm will be described hereinlater. 

[0228] Fig. 11 shows a schematic diagram of the MPEG-2 TS transcoder 200 used to describe the function and 
construction of the video transcoding unit 100. The constitutional elements enclosed by the broken line is the video 
transcoding unit 100 in Fig. 6. 

[0229] The video transcoding unit 100 is as shown in Fig. 11 comprising a VLD (Variable Length Decoder)51, an 
25 inverse quantizer 53, a quantizer 55, a VLC (Variable Length Encoder) 57, a virtual buffer management unit 105, a 
quantization scaling factor computing unit 107, a packetsAime number computing unit 111 , a target output video TS 
packet number computing unit 113, a reference 1/0-bit ratio computing unit 115, a TS/ES bit ratio computing unit 117, 
a bits/time difference computing unit 119 and a video ES buffer 243. 

[0230] The rate conversion method perfomied by the video transcoding unit 100 comprises the steps of: 



35 Step (1): Computing a reference ratio of outputting bits to inputting bits 

[0231] The reference ratio of outputting bits to inputting bits at time interval of the predetennined duration (n), herein- 
later refen'ed to as "ioRatio(n)" is computed on the basis of the target number of outputting bits of the output video 
elementary streams to be outputted at time interval of the predetemnined duration (n) and the number of real inputting 

40 bits of the video elementary streams actually inputted at time inten^al of the predetermined duration (n). 

[0232] The number of real inputting bitsof the video elementary streams inputted at time Interval of the predetennined 
duration (n), hereinlater referred to as " Video_E S_ln(n)", the number of the video TS packets inputted at time interval 
of the predetermined duration (n), hereinlater referred to as " Vldeo_TS_ln(n)", the number of non-reduction TS pack- 
ets, hereinlater referred to as "NonV_TSJn(n)", the total number of the input MPEG-2 TS packets inputted at time 

45 interval of the predetermined duration (n), hereinlater refenred to as "AILTSJn{n)", and the target number of the output 
video TS packets to be outputted at time interval of the predetemnined duration (n), hereinlater referred to as 
"tVldeo_TS_out(n)", the target number of the non-reduction TS packets to be outputted at time interval of the prede- 
termined duration (n), hereinlater referred to as, tNonV_TS_out(n), the target number of the output MPEG-2 TS pack- 
ets to be outputted at time Interval of the predetermined duration (n), hereinlater referred to as "tAILTS_out(n)", 

50 AILTS Jn(n) and tAILTS_out(n) are computed as follows: 



30 



(1) computing a reference ratio of outputting bits to inputting bits; 

(2) computing a reference quantization scaling factor; and 

(3) computing a quantization scaling factor in consideration of the re-quantization rate-distortion perfomnance. 



TSBin X bf^time . {n+ 1) 
188X8 



AIITSiJn) = 



55 



i=0 



...equation (44) 
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^ ...equation (45) 

where bMime (n) is time interval of he predetermined time duration (n), TSBin and TSBout are Input and output 
bit rates, respectively. 

10 [0233] Accordingly, the initial value of AILT SJn and tAII_TS_out i.e., AII_T S Jn(0) and tAILTS_out(0) are the 

first temn of the right side In the equations (44) and (45), respectively. 

[0234] AH_T SJn(n) and tAILTS_out(n) may not result in integers if calculated In accordance with only the first 
term of the right sides In the equations (44) and (45) with variables, i.e., TSBin and TSBout. 
[0235] Substitute, for instance, 4 [Mbps], for TSBout and, substitute, for instance, 1 [second], for bMlme (n) in the 
^5 equation (45) to calculate tAILTS_out(n); 



4000000/(1 88 X 8) = 2659.57 



20 

2659.57 — Is rounded down to 2659. 



[0236] Thus, tAILTS_out(n) results in 2659 [/second] 
[0237] This means; 

25 

tAILTS.out(n) is 2659 x 60 = 1 59540 [/minute] (A) 



[0238] On the other hand, substitute 4 [Mbps] for TSBout and, substitute 1 [minute] for bf_tlme (n) In the equation 
30 (45) to calculate tAILTS_out(n); 



(4000000 X 60)/(1 88 X 8) = 1 59574 

35 

Thus, tAILTS_out(n) results in 1 59574 [/minute] (B) 



[0239] Subtract the result (A) from the result (B) 

40 



15974-159540 = 34. (C) 



[0240] This means that the target number of the TS packets to be outputted at time interval of 1 [second], i.e., 
tAII_TS_out(n) is computed in accordance with the first term of the right sides in the equation (45) alone, an en^or of 

45 34 [packets/minute] results for every one minute. If the transcoder 200 is operated to transcode the M PEG-2 TS packets 
for a two-hour movie, the error of 34 [packets] increases to 2040 [packets/hour], which is critical for the operation. To 
eliminate such en-or, the equations (44) and (45) have the second terms in the right side, respectively 
[0241] Vldeo_TSJn(n) and NonV_TSJn(n) can be obtained when the input TS packets are demultiplexed. 
[0242] As described hereinbefore, newly generated PAT, PMT may be transmitted from the PAT, PI^T generator 260 

50 to the output MPEG-2 TS muitiplexer 220 at time interval of the predetermined duration (n). tNonV_TS_out{n) can be 
calculated as follows: 



tNonV_TS_out(n)=NonV_TS_in(n) + PATPMT_out(n) equation (46) 

55 

where PATPMT_out(n) has been described in the aforesaid equation (38). 
[0243] The difference between the target number of the video TS packets and the number of real outputted video 
TS from the time Interval of the predetennlned duration (0) to the time interval of the predetermined duration (n-1), 
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hereinlater referred to as "dVldeo_TS(n)", tVldeo_TS_out(n) is calculated as follows: 

tVideo_TS_out(n) = tAILTS_out(n) — tNonV_TS_out(n)+dVideo_TS(n) equation (47) 

5 

dVideo_TS(n) is calculated as follows: 

10 dVideoTSin) = 2 tVideoTS^utU) - totalVideoTS^^t 

' ° ...equation (48) 

where the initial value of dVldeo_TS, i.e., dVldeo_TS(0) = 0, tVldeo_T S_out{l) is the number of TS packets 
15 actually generated at time interval of the predetennined duration (1), totaLVIdeo_TS_out is the total number of TS 
packets which have been generated by the video TS packet generator 246 until the time when the video ES buffer 243 
consumes all the video elementary streams Inputted from (0) to (n-1). 

tVldeo_TS_out(n) is computed in accordance with the equation (47) by a target output video TS packet number com- 
puting unit 113, which is shown in Fig. 11 . The target output video TS packet number computing unit 11 3 constitutes 
20 the computing units (A) and (B). Here, tNonV_TS_out(n) serves as the value A, and dVldeo TS(n) serves as the value 
B, tVldeo_TS_out(n) serves as the target number of outputting bits of the transformed first data string. 
[0244] Accordingly, the reference ratio of outputting bits to inputting bits at time Interval of the predetemiined duration 
(n), i.e., ioRatlo(n), is calculated as follows: 



25 



tVideoTS^inx ^88 



30 where output_TE_Ratlo(n) is a bit ratio between the number of real outputting bits of the video elementary 

streams, I.e., the video ES packets, and the number of real outputting TS packets. ioRatio(n) is calculated in accord- 
ance with the equation (49) by the reference 1/0-bit ratio computing unit 115, which is shown in Fig. 1 1 . The reference 
l/O-bit ratio computing unit 115 constitutes the computing units (C), (E), (F), and (G). 

[0245] Overhead is generated while the video PES packet generator 245 and the video TS packet generator 246 Is 
35 operated to packetize the video elementary streams to generate the video transport streams from time Interval of the 
predetermined duration (0) to time interval of the predetermined duration (n-1). It is difficult to calculate the exact 
overhead. Accordingly, tVideo_TS_out(n) is divided by output_TE_Ratio(n) In the equation (49) to minimize the over- 
head. 

output_TE_Ratio(n) Is calculated as follows: 

40 

XVi<i^oTS^^^{i) X188 

45 ou tp u tTERatio {n)= ^" ° ^ 

^VideoES^^^ii) 

...equation (50) 

50 

output_TE_Ratlo(n) is computed in accordance with the equation (50) by the reference I/O bit ratio computing 
unit, which Is shown In Fig. 11 . The reference I/O bit ratio computing unit constitutes the computing unit (F). 
[0246] The initial value of output_TE_Ratlo, i.e., output_TE_Ratio(0) cannot be calculated with the equation (50). 
output_TE_Ratlo(0) is specified in the following manner. The transcoder starts transcoding the video bit streams with 
55 the sequence header. As described hereinbefore, the video PES packet decoder 242 is operated to detect the first 
sequence header upon transcoding the video bit streams and delete bit streams prior to the first sequence header If 
there are provided bit streams prior to the first sequence header. This means that the l-picture Is firstly processed. If 
bMlnfie is small, the l-plcture may not an^ive at the video ES buffer 243. Provided that the hplcture does not arrive at 
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the video ES buffer 243, the overhead thus resulted consists of the TS packet header (4 bytes) of each TS packet, the 
PES packet header of the head PES packet (9 bytes), the PTS (5 bytes) and the DTS (5 bytes). The overhead Is 
calculated as follows: 



tVideoTS^(0)x^SS 
^^^^^™^^^Q)= fWdeorS,^(0)x184-19 ^^^^'^'^ (^^) 

[0247] Otherwise, output_TE_Ratio(0) can be specified to be, for instance, 1 .03 by experiments 

[0248] The target number of bits of the output video elementary stream, hereinlater referred to as "tVideo_ES_out 

(n)", Is calculated as follows: 

tVideoES^r^ ouputTERatio(n) (52) 

tVldeo_ES_out(n) Is computed in accordance with the equation (42) by the reference I/O bit ratio computing 
unit, which Is shown In Fig. 11 . The reference I/O bit ratio computing unit constitutes the computing unit (F). 
[0249] Threshold value, i.e., TH_out(n) Is calculated as follows: 



TH^ui(n)= tVideoES^J^n) + totalVideoES^ equation (53) 

where total_Vldeo_ES_out is the total bit number of the video elementary streams which have been generated 
until the time when video elementary streams inputted at time Interval of (n-1) into the video ES buffer 243 are con- 
sumed. TH_out (n) is substituted in the equation (43). TH_out(n) is calculated by the bits/time difference computing 
unit 1 1 9, which is shown In Fig. 1 1 . The bits/time difference computing unit 11 9 constitutes the computing units (H) and 
(I), the judging unit (A) and the control unit. 

[0250] As described hereinbefore, TH_out(n) is the reference threshold value of the transcoding operation and, here, 
the expected value of the total byte number of the video elementary streams outputted from the time interval of the 
predetermined duration (0) to the time interval of the predetermined duration (n). Beside the expected value of the 
total byte number of the video elementary streams outputted from the time interval of the predetermined duration (0) 
to the time interval of the predetemnined duration (n), other values can be considered to be the reference threshold 
value, TH_out(n). As the reference threshold value of the transcoding operation, I.e., TH_out(n), following two values 
can be assumed. 

Case(1); 

[0251] Assume the time at which the video ES buffer 243 consumes all the input video elementary streams inputted 
at time interval of the predetemriined duration (n) as the reference threshold value of the transcoding operation i e 
TH^out(n). 

The drawback of the case (1): 

[0252] If the bit rate is not exactly controlled at time interval of the predetermined duration (n), an error between the 
target number of outputting video TS packets and the number of real outputting video TS packets at time Interval of 
the predetermined duration (n) may increase. 

Case (2): 

[0253] Assume the time at which the target number of outputting video TS packets matches the number of real 
outputting video TS packets as the reference threshold value of the transcoding operation, i.e., TH_out(n). 
The drawback of the case (2): 

[0254] If the bit ratio between the number of the video ES packets and the number of the video TS packets at time 
Interval of the predetennined duration (n) Is smaller than the bit ratio between the number of real outputting bits of the 
video ES packets and the number of real outputting TS packets, i.e., output_TE_Ratlo(n) at time interval of the pre- 
determined duration (n), part of the video ES packets may delay, and In such case time interval of the predetermined 
duration (n) elapses before the target number of the video ES packets are outputted. 

[0255] The equation (53) has merits of being able to :(1 ) tolerate the error of the bit rate control; and (2) perfonn the 
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time-base compensation. 

Step (2): Computing a reference quantization scaling factor 

[0256] Compute the quantization scaling factor required for transfomning the video elementary streams to generate 
said output video elementary streams, on the basis of the reference ratio of outputting bits to inputting bits at time 
interval of the predetermined duration (n), i.e., ioRatio(n) calculated by the equation (49). 

[0257] The quantization scaling factor is computed while MBs are sequentially encoded at time interval of the duration 
(n). The number of bits and the total number of bits are referred to as the "bit number" and the "total bit number", 

respectively, 

[0258] The quantization scaling factor is computed through the steps of: 

(a) computing the total bit number of MB inputted and decoded at time interval of the predetermined duration (n); 

(b) computing the estimated total bit number of encoded MB, i.e., the estimated bit number of encoded MB to be 
stored in the virtual buffer, hereinlater referred to estimated "utilization volume" of the virtual buffer on the basis of 
the product of the total bit number of MB and ioRatio(n) calculated by the equation (49); and 

(c) compute the quantization scaling factor In consideration of the difference between the estimated utilization 
volume of the virtual buffer and the total bit number of real outputting encoded MB, i.e., utilization volume of the 
output buffer. 

[0259] Before transcoding a j-th MB at time interval of the predetermined duration (n), the utilization volume of the 
virtual buffer is computed as follows: 



(;)= b(0) + rc X 2, \BMBout{k) " (k)XioRatio (n) \ 

...equation (54) 

where B_MB_out(k) is the bit number of k-th MB (k) in the output bit stream at time interval of the predetermined 
duration (n) and B_MB_ln(k) Is the bit number of k-th MB (k) in the Input bit stream at time Interval of the predetermined 
duration (n), rc is a delay control parameter 

[0260] The reference quantization scaling factor is computed as follows: 

QC)= 31 X b G)/ r equation (54) 

[0261] In the equation (54), b(0) is the initial utilization volume of the virtual buffer. The total number of MBs is 
hereinlater referred to as "MB_cnt". b(MB_cnt) at time Interval of the predetermined duration (n) Is substituted for b 
(0) of the virtual buffer at time Interval of the predetennined duration (n+1). The reference quantization scaling factor 
is computed in accordance with the equation (55) by the reference 1/0-bit ratio computing unit 115, which is shown In 
Fig. 1 1 . The reference l/O-bIt ratio computing unit 115 constitutes the computing unit (D). 
[0262] In the equation (55), r is calculated as follows: 

r = 2 X tVldeo_ES_out(n) equation (56) 

[0263] The Initial value of the b(0), I.e. , b(0) at time interval of the predetennined duration (0) Is calculated as follows: 

b(0)= (lOXr) /31 equation (57) 

Step (3): Computing the quantization scaling factor in consideration of the rate-distortion performance 
[0264] The quantization scaling factor is calculated on the basis of the reference quantization scaling factor, i.e., Q 
0), computed In the Step (2) in consideration of the re-quantization rate-distortion performance so as to minimize the 
quantization error. The quantization scaling factor for the j-th macroblock, hereinlater referred to as "mq(])", Is calcu- 
lated in the following equations. Here, the detailed description of the equation is omitted. 
In the case of intra-MB: 
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10 



IS 



25 



QU)-l 



...equation (58) 



In the case of inter-MB: 



mq(j)=mqi„ {j)x 



1 






2 . 




2 



+ 1 

..equation (59) 



where the symbol of is Intended to mean the rounding operation, and mc|_ln(j) is the quantization scaling factor 
computed for the decoded j-th macroblock (j) In the Input bit stream. 
[0265] In the case of mQ(j)<mculn(j): mQ(j) = mq_ln(j) 
20 [0266] The equations (58) and (59) minimize the re-quantization rate-distortion performance occurred while the re- 
quantization and quantization operations. 

[0267] The J-th macroblock (j) is quantized with the mQ(]) thus computed. 

[0268] The utilization volumes of video and audio buffers when the output MPEG-2 TS transcoded by the transcoder 
Is inputted to a T-STD (System Target Decoder: avirtual decoder model) will be described hereinlater. 



Experimentat simulation result (1) 



[0269] The transitions of utilization volumes of the video and audio buffers in the T-STD for the output bit streams 
and the Intput bit streams were measured under the condition that the Input MPEG-2 bit stream of "stream A" was 
30 transcoded by the transcoder 200 with the target bit rate of 4 [Mbps], and inputted to the T-STD, and under the condition 
that the same Input MPEG-2 bit stream was directly Inputted to the T-STD without the transcoding operation, respec- 
tively. The both measurements were performed at time inten/al of the predetemfilned duration (n), I.e., bf_tlme[8ec] 
is 0.02. 

[0270] The buffer utilization volume was always updated through the steps of (1) through (5) as follows: 

35 

(1) compute the current time on the basis of the PCR, the input bit rate, and the bit number in the Input bit stream; 

(2) detect the PIS (or DTS) of the oldest frame in the buffer; 

(3) judge If the cun^ent time is equal to the PTS (or DTS) of the oldest: 

(4) eliminate the oldest frame If It Is judged that the current time is equal to the PTS (DTS) of the frame, and 
40 (5) compute the number of bits contained in the buffer. 

[0271] Fig. 12 shows the transition of the T-STD video elementary buffer utilization volume. Fig. 12 (a) shows the 
transition of the T-STD video elementary buffer utilization volume for the Input bit streams and Fig. 12(b) shows the 
transition of the T-STD video elementary buffer utilization volume for the output bit streams. 
45 [0272] The horizontal axis is intended to mean a frame number, i.e. frame No. in the order of the frame was decoded, 
and the vertical axis Is Intended to mean the utilization volume of the video elementary buffer. The dotted line is Intended 
to mean the buffer size. 

[0273] Fig. 13 shows the transition of the utilization volume of the T-STD audio buffer. Fig. 13 (a) shows the transition 
of the T-STD audio elementary buffer utilization volume for the Input bit streams and Fig. 13 (b) shows the transition 

50 of the T-STD audio elementary buffer utilization volume for the output bit streams. 

[0274] The horizontal axis Is intended to mean an audio frame number, i.e. audio frame No. in the order of the frame 
was decoded, and the vertical axis Is intended to mean the utilization volume of the audio buffer. The dotted line Is 
intended to mean the buffer size assigned to the audio elementary streams and PES packet header buffering. For the 
sake of clarity, the horizontal axis Is In the range of frame No. 0 to frame No. 30 In Fig. 13 while it is in the range of 

55 frame No. to frame No. 1200. 

[0275] As shown in Fig. 12, the transition of the T-STD video elementary buffer utilization volume for the output bit 
streams occurs in proportion to that of the T-STD video elementary buffer utilization volume for the Input bit streams. 
This results from the fact that the bit number of one frame in the video ES contained in the input bit streams Is propor- 
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tlonal to that of the same frame In the video ES contained In the output bit streams. 

[0276] As shown in Fig. 13, the transition of the T-STD audio elementary buffer utilization volume for the output bit 
streams occurs in proportion to that of the T-STD audio elementary buffer utilization volume for the Input bit streams. 
This results from the fact that the bit streams constituting an audio frame contained In the bit streams arrive at the T- 
5 STD audio buffer at the same time at which bit streams constituting the same audio frame contained in the input bit 
streams are supposed to arrive at the T-STD audio buffer. This means that the output bit stream would not overflow 
the audio buffer unless the Input bit stream does not overflow the audio buffer 

[0277] The foregoing Experimental simulation proves that the transcoder 200 is operated to successfully perfomi 
the demultiplexing operation on the basis of the synchronous information elements contained in the input MPEG-2 
10 transport streams. 

Experimental simulation result (2) 

[0278] The transitions of utilization volumes of the video and audio buffers in the T-STD for the output bit streams 
IS and the Intput bit streams were measured under the condition that the input MPEG-2 bit stream of "stream B" was 
transcoded by the transcoder 200 with the target bit rate of 4 [Mbps], and inputted to the T-STD, and under the condition 
that the same Input MPEG-2 bit stream was directly Inputted to the T-STD without the transcoding operation, respec- 
tively. The both measurements were perfomried at time inten/al of the predetennined duration (n), i.e., bMime[sec] 
is 0.02. 

20 [0279] Fig. 1 4 is the transition of the T-STD video elementary buffer utilization volume. Fig. 1 4 (a) shows the transition 
of the T-STD video elementary buffer utilization volume for the Input bit streams and Fig. 12(b) shows the transition of 
the T-STD video elementary buffer utilization volume for the output bit streams. The meanings of the horizontal axis 
and vertical axis are same as those In Fig. 12. 

[0280] Fig. 1 5 shows the transition of the utilization volume of the T-STD audio buffer. Fig. 1 5 (a) shows the transition 
25 of the T-STD audio elementary buffer utilization volume for the input bit streams and Fig. 15 (b) shows the transition 
of the T-STD audio elementary buffer utilization volume for the output bit streams. The meanings of the horizontal axis 
and vertical axis are same as those In Fig. 13. 

[0281] As shown in Fig. 14, the transition of the T-STD video elementary buffer utilization volume for the output bit 
streams occurs in proportion to that of the T-STD video elementary buffer utilization volume for the Input bit streams. 
30 This results from the fact that the bit number of one frame In the video ES contained in the input bit streams is propor- 
tional to that of the same frame in the video ES contained in the output bit streams. 

[0282] As shown in Fig. 15, the transition of the T-STD audio elementary buffer utilization volume for the output bit 
streams occurs in proportion to that of the T-STD audio elementary buffer utilization volume for the input bit streams. 
Fig. 1 5 (b), however, shows some audio frames having extremely high utilization volumes. The audio buffer may cause 
55 overflow with large audio frames contained in the input bit streams while the audio buffer do not overflow with those 
audio frames yet in this Fig. 15(b). 

[0283] Fig. 16 shows input and output MPEG-2 TS which may cause the buffer overflow. The horizontal axis Is 
Intended to mean the time axis. The input MPEG-2 TS packets and the output MPEG-2 TS packets are shown in the 
upper and lower sections, respectively. The output bit rate of the output MPEG-2 TS is reduced to the half of the Input 

40 bit rate of the input MPEG-2 TS. The TS packets with sloped lines are intended to mean the audio TS packets. 

[0284] The audio TS packets which may cause the buffer overflow are audio TS packets A and B in Fig. 16. The 
whole input audio TS packet A arrive at the audio buffer at the same time, i.e., t2 at which the whole output audio TS 
packet A arrives at the audio buffer. While on the contrary, the first byte of the output audio TS packet A arrives at the 
audio buffer at tO eariier than t1 at which the input audio TS packet A arrives at the audio buffer. On the other hand, 

45 the first byte of the output audio TS packet B arrives at the audio buffer at the same time, I.e.. tS at which the first byte 
of the output TS packet B arrives at the audio buffer. While on the contrary, the whole Input audio TS packet arrives 
at the audio buffer at t4 eariier than tS at which the input audio TS packet A arrives at the audio buffer. 
[0285] A seen from Fig. 16, although the audio buffer is full with bit streams before the first byte of the input audio 
TS packet A arrives at the audio buffer, the bit streams which contain the synchronous infonnation element, i.e. PTS, 

50 are removed at time indicated by the PTS. This means that the whole input audio TS packet A arrives at the audio 
buffer after the bit streams which contain the PTS have been removed and, therefore, does not cause the buffer over- 
flow. On the contrary, the first byte of the output audio TS packet A arrives at the audio buffer at tO i.e., before the time 
Indicated by PTS. This means that the whole output audio TS packet A arrives at the audio buffer before the bit streams 
which contain the PTS have not been removed and, therefore, may cause the buffer overflow. 

55 [0286] In the case of the audio TS packet B, bit streams which contain the synchronous information element, i.e., 
PTS' are located posterior to the audio TS packet B. The whole input audio TS packet B arrives at the audio buffer at 
t4, prior to the time indicated by PTS* and will be decoded by the time indicated by PTSV On the contrary, the whole 
output audio TS packet B does not arrive yet at the time indicated by PTS', but an^ives at the audio buffer at tS, posterior 
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to the time indicated by PTS'. This means that some parts of the output audio TS packet B may remain not be decoded 
at the time indicated by PTSV The phenomenon of the audio TS paclcet B is hereinlater referred to "underflow", and 
can take place more likely as the bit rate ratio of the target bit rate to the input bit rate increases. 
[0287] In Fig. 15 (b), some audio frames have extremely high utilization volumes. This results from the fact that the 
5 input bit stream B contains audio TS packets of the type such as the audio TS packet shown in Fig. B, which increase 
buffer utilization volumes. 

[0288] In order to prevent the buffer overflow from taking place, there Is provided a packet relocating method com- 
prising the steps of: 

10 (1 ) computing a difference by subtracting a value of lastly past presentation time management Infonmation element, 

i.e., Iast_PTS, of the first coded signal, i.e., input MPEG-2 transport streams, from a value of a synchronous time 
information element, i.e., PCR_current_audlo, located in a head position of a data string of the one or more 
second data strings, i.e., an audio TS packet demultiplexed by the output MPEG-2 TS packet demultiplexer; 

(2) computing a difference between passing time of said data string of said one or more second data strings, i.e. 
15 an audio TS packet, in the input MPEG-2 transport streams and passing time of the audio TS packet in the output 

MPEG-2 transport streams; 

(3) judging upon whether said difference computed In step (1) Is smaller than said difference computed In the step 
(2); and 

(4) locating the audio TS packet in a reanward position of a data string of said one or more first data strings, i.e. a 
20 video TS packet, to be located in a reanA/ard position of the audio TS when it is judged that the difference computed 

In the step (1) is smaller than the difference computed in the step (2). 

[0289] The difference between last_PTS and PCR_current_audlo is compared with the difference between the 
passing time of the audio TS packet In the input MPEG-2 transport streams and the passing time of the audio TS 
25 packet In the output MPEG-2 transport streams In accordance with the equation as follows: 

PCRcurrent_audio — last^PTS < audiojh equation (60) 

30 where audlo^th is the difference between the passing time of the audio TS packet in the input MPEG-2 transport 

streams and the passing time of the audio TS packet in the output MPEG-2 transport streams and is computed as 
follows: 

35 . TSB:„ V 

(^^^g^~l)X188X8 

^udio th = '-^ X 27000000 

TSB^n 

40 ...equation (61) 

[0290] On the other hand, in orderto prevent the buffer underflow, there Is provided a method comprising the steps of: 

45 (1) computing a difference by subtracting a value of a synchronous time infomriation element to be past subse- 

quently in the input MPEG-2 transport streams, I.e., PCR_current_audio, from a value of a presentation time 
management information element of a just past audio TS packet in said second coded signal, i.e., next_PTS; 

(2) computing a difference between passing time of said data string of said one or more second data strings, i.e. 
an audio TS packet, In the input MPEG-2 transport streams and passing time of the audio TS packet in the output 

50 MPEG-2 transport streams; 

(3) judging upon whether the difference computed in the step (1) is smaller than the difference computed in the 
step (2); and 

(4) locating the video TS packet In a fonward position of a data string of said one or more first data strings, i.e. a 
video TS packet, to be located in a forward position of the audio TS packet when it is judged that the difference 

55 computed in the step (1) is smaller than the difference computed in the step (2). 

[0291 ] The difference between PCR_current.audio and next_PTS is compared with the difference the passing time 
of the audio TS packet in the input MPEG-2 transport streams and the passing time of the audio TS packet in the output 
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MPEG-2 transport streams in accordance with the equation as follows: 

next_PTS — PCR_cun'ent_audio< audiojh equation (62) 

5 

where audio_th is computed with the equation (61). 
[0292] PCRcurrent.audio serves as a value of a synchronous time Infomnation element PGR located in a head 
position, last^PTS serves as a value of lastly past presetntation time management information element PTS. 
PCRcurrent_audio-last_PTS is computed by the computing unit (J). next_PTS serves as a value of a presentation 
10 time management infomiation element PTS of a just past data string, and PCR_current_audio serves as a synchro- 
nous time information element PGR to be past subsequently. next_PTS-PGR_current_audio is computed by the 
computing unit (L). audiojh is computed by the computing units (K) and (M). Computing units (J). (K), (L), and (M) 
are constituted by the bit/time difference computing unit 119. The Judging units (B) and (C), and locating unit (A) and 
(B) are constituted by the output MPEG-2 TS multiplexer 220. 

15 

Experimental simulation result (3) 

[0293] The transitions of utilization volumes of the video and audio buffers in the T-STD for the output bit streams 
were measured under the condition that the input MPEG-2 bit stream of the stream B was transcoded by the transcoder 
20 200 which Implements the aforesaid packet relocating method, with the target bit rate of 4 [l\/Ibps], and inputted to the 
T-STD. 

[0294] Fig. 17 shows the transition of the utilization volume of the T-STD video buffer and the T-STD audio buffer. 
Fig. 1 7 (a) shows the transition of the T-STD video elementary buffer utilization and Fig. 1 7 (b) shows the transition of 
the T-STD audio elementary buffer utilization volume. 
25 [0295] As will be understood from Fig. 1 7(b) and Fig. 1 5, the audio frames having extremely high utilization volumes 
(shown in Fig. 15) are eliminated in Fig. 17(b). Furthermore, the transition of the T-STD video and audio elementary 
buffer utilization volume for the output bit streams occurs in proportion to that of the T-STD video and audio elementary 
buffer utilization volume for the input bit streams. Accordingly, the effectiveness of the packet relocating method is 
verified, 

30 [0296] The present invention provide an apparatus, a method, and a computer program product for transcoding a 
coded multiplexed sound and moving picture sequence, which demultiplexs the first coded signal Into one or more first 
data strings, one or more second data strings and one or more third data strings, and compresses only the one or 
more first data strings to generate one or more compressed first data strings, and multiplexes the one or more com- 
pressed first data strings, the one or more second data strings, and the one or more third data strings into the second 

35 coded signal, thereby making it possible to reduce the amount of the related operations, and the operation time. 

[0297] The present invention provide an apparatus, a method, and a computer program product for transcoding a 
coded multiplexed sound and moving picture sequence, which is targeted for compressing only the video bit stream, 
which is regarded to have extremely large infomnation volume in the MPEG-2 transport, thereby making it possible to 
effectively control the target bit rate. 

40 [0298] The present invention provides an apparatus, a method, and a computer program product for transcoding a 
coded multiplexed sound and moving picture sequence, which sets the system clock indicative of the start time of the 
operation on the basis of the first reference time infomnation element contained in the first coded signal, thereby making 
It possible to synchronize the system clock of the first coded signal with that of the second coded signal. 
[0299] It will be apparent to those skilled in the art and it is contemplated that variations and/or changes in the 

45 embodiments illustrated and described herein may be without departure from the present invention. Accordingly, it is 
intended that the foregoing description is illustrative only, not limiting, and that the true spirit and scope of the present 
invention will be determined by the appended claims. 



50 Claims 

1. A method of transcoding a coded multiplexed sound and moving picture sequence, comprising the steps of; 

(a) inputting a first coded signal through a first transmitting path at an input bit rate; 
5S (b) demultiplexing said first coded signal inputted In said inputting step (a) into one or more first data strings, 

one or more second data strings, and one or more third data strings, said one or more first data strings each 
having a number of real inputting bits; 

(c) transfomning said one or more first data strings demultiplexed in said demultiplexing step (b) into one or 
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more first data strings having a number of real outputting bits less than said number of real inputting bits of 
said one or more first data strings, respectively; 

(d) multiplexing said one or more transformed first data strings transformed in said transfonning step (c), said 
one or more second data strings demultiplexed in said demultiplexing step (b), and one or more corrected 
third data strings to generate a second coded signal; 

(e) correcting said one or more third data strings demultiplexed in said demultiplexing step (b), on the basis 
of said first coded signal, in accordance with a change of said second coded signal to generate said one or 
more corrected third data strings having a number of real inputting bits when said one or more transformed 
first data strings, said one or more second data, and said one or more corrected third data strings are multi- 
plexed in said multiplexing step (d); and 

(f) outputting said second coded signal through a second transmitting path at a target bit rate lower than said 

Input bit rate of said first coded signal. 

The method as set forth in claim 1 , wherein said inputting step (a) has the step of {a2) inputting MPEG-2 transport 
streams, said demultiplexing step (b) has the step of (b2) demultiplexing said l\/IPEG-2 transport streams Inputted 
in said inputting step {a2) Into one or more transport stream packets having a coded video signal having a number 
of real Inputting bits as one or more first data strings, and said outputting step (f) has the step of (f2) outputting 
MPEG-2 transport streams having a video signal having a number of real outputting bits less than said number of 
real inputting bits of said coded video signal. 

The method as set forth in claim 1 , wherein said inputting step (a) has the step of (a3) inputting coded multiplexed 
sound and moving picture sequence streams, said demultiplexing step (b) has the step of (b3) demultiplexing said 
coded multiplexed sound and moving picture sequence streams inputted in said inputting step (a3) Into one or 
more transport stream packets having a coded moving picture sequence signal having a number of real inputting 
bits as said one or more first data strings, and said outputting step (f) has the step of (fS) outputting coded multi- 
plexed sound and moving picture sequence streams having a coded moving picture sequence signal having a 
number of real outputting bits less than said number of real Inputting bits of said coded moving picture sequence 
signal. 

The method as set forth in claim 2 or claim 3, which further comprises the steps of: 

(g) computing a value of system clock Indicative of a start time of said demultiplexing step (b), on the basis of 
a first reference time information element contained in said first coded signal; and 

(h) computing an Initial value of said system clock for said second coded signal on the basis of said value of 
said system clock computed in said computing step (g). 

The method as set forth in any claim of claims 2 through 4, wherein said transforming step (c) further comprises 
the steps of: 

(c51) decoding said one or more transport stream packets having a coded video signal having a number of 
real Inputting bits to reconstruct and output video PES packets; 

(c52) decoding said video PES packets decoded in said decoding step (c51) to reconstmct and output video 
elementary streams having a real inputting bits, decoding time management infomriation element DTS and 
presentation time management information element PTS of said video elementary streams, and PTS_DTS 
flags indicative of presence of said decoding time management Infonnation element DTS and said presentation 
time management Infonnation element PTS; 

(c63) transforming said video elementary streams decoded and outputted in said decoding step (c52) into 
video elementary streams having a number of real outputting bits less than said number of real inputting bits 
of said video elementary streams; 

(c54) generating transfomied video PES packets on the basis of said transfomned video elementary stream 
transformed in said transforming step (c53), said decoding time management infonnation element DTS, said 
presentation time management Infonnation element PTS and said PTS_DTS flags indicative of presence of 
said decoding time management information element DTS and said presentation time management informa- 
tion element PTS decoded In said decoding step (c52); and 

(c55) encoding said transfomned video PES packets generated in said generating step (c54) to generate a 
transfomied video transport stream packets having a number of real outputting bits less than said number of 
real Inputting bits of said video transport stream packets. 
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6. The method as set forth in any claim of claims 2 through 5, wherein said demultiplexing step (b) has the step of 
demultiplexing said first coded signal into transport stream packets having a coded audio signal as said one or 
more second data strings. 

7. The method as set forth in any claim of claims 2 through 6, wherein said demultiplexing step (b) has the step of 
{b71 ) demultiplexing MPEG-2 transport streams inputted at a predetermined time interval in said inputting step (a) 
into said one or more first data strings, said one or more second data strings, and said one or more third data 
strings, each having a number of real inputting bits, and said transfonning step (c) comprises the steps of: 

(c71) decoding said one or more first data strings to reconstruct video elementary streams having a number 
of real Inputting bits and other inf omiation elementary streams, and separating said video elementary streams 
from said other infomnation elementary streams; 

(c72) transforming said video elementary streams reconstructed and separated In said decoding step (c71) 
to generate output video elementary streams having a number of real outputling bits less than said number 
of real inputting bits of said video elementary streams; and 

(c73) generating one or more transformed first data strings having a number of real outputting bits less than 
said number of real inputting bits of said one or more first data strings demultiplexed in said demultiplexing 
step (b71) on the basis of said output video elementary streams generated In said transfonning step (c72) 
and said other information elementary steams reconstructed and separated in said decoding step (c71 ), where- 
by said multiplexing step (d) has the step of (d71) multiplexing said one or more transfomned first data strings 
generated in said generating step (c73), said one or more second data strings demultiplexed in said demulti- 
plexing step (b71), and 

said one or more corrected third data strings con-ected In said correcting step (e) to generate a second coded 
signal to be outputted at said predetemirned time interval. 

8. The method as set forth in claim 7, wherein said transforming step (c) comprises the steps of: 

(c81 ) assuming that a number of real outputting bits of said one or more second data strings contained in said 
second coded signal at said predetermined time Interval is equal to a number of real inputting bits of said one 
or more second data strings contained in said first coded signal at said predetemrjined time interval; 
(c82) assuming that a number of real inputting bits of said one or more third data strings contained in said 
second coded signal at said predetermined time interval is equal to said number of real outputting bits of said 
one or more corrected third data strings generated in said correcting step at said predetermined time interval; 
(c83) subtracting said number of real inputting bits of said one or more second data strings contained in said 
first coded signal at said predetennined time interval and said number of real inputting bits of said one or more 
third data strings contained in said second coded signal at said predetermined time interval from a target 
number of outputting bits of all data strings contained in said second coded signal at said predetermined time 
interval to generate a value A; 

(c84) subtracting a total number of real outputting bits of one or more transformed first data strings generated 
in said generating step {c73) prior to said predetennined time inten^al from a total target number of outputting 
bits of one or more transfomned first data strings generated in said generating step (c73) prior to said prede- 
tennined time interval to generate a value 8; and 

{c85) computing a target number of outputting bits of said transfomned first data strings generated at said 
predetennined time interval in said generating step (c73) by adding said value A and said value B. 

9. The method as set forth in claim 7 or claim 8, wherein said transfonning step (c72) comprises the steps of: 

(c91) computing a target number of outputting bits of said output video elementary streams on the basis of 
outputting bits of said second coded signal able to be outputted at said predetennined time interval; 
(c92) computing a reference ratio of outputting bits to inputting bits on the basis of said target number of 
outputting bits of said output video elementary streams computed In said computing step (c91 ) and said number 
of real inputting bits of said video elementary streams reconstructed in said decoding step (c71); and 
(c93) computing a quantization scaling factor required for transforming said video elementary streams to gen- 
erate said output video elementary streams, on the basis of said reference ratio of outputting bits to inputting 
bits computed in said computing step (c92). 

10. The method as set forth in claim 9, wherein said computing step (c91 ) comprises the steps of: 
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(c101) computing a ratio of a total number of real outputting bits of said first one or more transformed data 
strings generated in said generating step (c73) prior to said predetenmined time interval to a total number of 
real outputting bits of said output video elementary streams prior to said predetennined time interval; and 
(c102) computing a target number of outputting bits of said output video elementary streams at said prede- 
tennined time interval on the basis of said ratio computed in said computing step (c101), and said computing 
step (c92) has the step of (c103) computing a reference ratio of outputting bits to inputting bits on the basis 
of said target number of outputting bits of said output video elementary streams computed in said computing 
step (c1 02) and said number of real inputting bits of said video elementary streams decoded in said decoding 
step (c71). 

11. The method as set forth in any claim of claims 7 through 10, wherein said transforming step (c) comprises the 
steps of: 

(c1101) computing a total number of real outputting bits of said output video elementary streams generated 

in said transfomiing step (c72); 

(c1 1 02) computing a sum of a target number of outputting bits of said output video elementary streams at said 
predetemnined time Interval and a total number of real outputting bits of said output video elementary streams 
which have been generated until the time when video elementary streams inputted prior to said predetermined 
time interval into a video ES buffer are consumed; 

(c1 1 03) judging upon whether said total number of real outputting bits of said output video elementary streams 
computed in said computing step (c1101 ) is greater than said sum computed in said computing step (c1 102); 
and 

(C1104) temiinating said transforming step (c) and starting said multiplexing step (d) for processing said one 
or more transformed first data strings when it is judged that said total number of real outputting bits is greater 
than said sum in said judging step (c1103). 

12. The method as set forth in any claim of claims 2 through 1 1 , wherein said multiplexing step (d) comprises the steps 

of: 

(d1 201 ) computing a difference by subtracting a value of lastly past presentation time management infonnation 
element PIS of said first coded signal from a value of a synchronous time infomnation element PCR located 
in a head position of a data string of said one or more second data strings; 

(d1202) computing a difference between passing time of said data string of said one or more second data 
strings in said first coded signal and passing time of said data string of said one or more second data strings 
in said second coded signal; 

(d1203) judging upon whether said difference computed in said computing step (d1201) is smaller than said 
difference computed in said computing step (d1202); and 

(d1 204) locating said data string of said one or more second data strings in a reanward position of a data string 
of said one or more first data strings to be located in a rearward position of said data string of said one or more 
second data strings when it is judged that said difference computed in said computing step (d1201) is smaller 
than said difference computed in said computing step (d1202) in said judging step (d1203). and said outputting 
step (f) has the step of (f1201) outputting said second coded signal at said predetemnined time interval. 

13. The method as set forth in any claim of claims 2 through 1 2, wherein said multiplexing step (d) comprises the steps 
of: 

(d1 301) computing a difference by subtracting a value of a synchronous time information element PCR to be 
past subsequently in said first coded signal from a value of a presentation time management information 
element PTS of a just past data string of said one or more second data strings in said second coded signal; 
(d1302) computing a difference between passing time of said data string of said one or more second data 
strings in said first coded signal and passing time of said data string of said one or more second data strings 
in said second coded signal; 

(d1303) judging upon whether said difference computed in said computing step (d1301) is smaller than said 
difference computed in said computing step (d1302); and 

(d1 304) locating said data string of said one or more second data strings in a fonward position of a data string 
of said one or more first data strings to be located in a forward position of said data string of said one or more 
second data strings when it is judged that said difference computed in said computing step (d1 301 ) is smaller 
than said difference computed in said computing step (d1 302) in said judging step (d1 303), and said outputting 
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step (f) has the step of (f1301) outputting said second coded signal at said predetennined time interval. 

An apparatus of transcoding a coded multiplexed sound and moving picture sequence, comprising: 

inputting means for inputting a first coded signal through a first transmitting path at an input bit rate; 
demultiplexing means for demultiplexing said first coded signal inputted by said inputting means into one or 
more first data strings, one or more second data strings, and one or more third data strings, said one or more 

first data strings each having a number of real inputting bits; 

transfonning means for transforming said one or more first data strings demultiplexed by said demultiplexing 
means into one or more first data strings having a number of real outputting bits less than said number of real 
inputting bits of said one or more first data strings, respectively; 

multiplexing means for multiplexing said one or more transfomned first data strings transformed by said trans- 
forming means, said one or more second data strings demultiplexed by said demultiplexing means, and one 
or more corrected third data strings to generate a second coded signal; 

correcting means for correcting said one or more third data strings demultiplexed by said demultiplexing 
means, on the basis of said first coded signal, in accordance with a change of said second coded signal to 
generate said one or more corrected third data strings having a number of real inputting bits when said one 
or more transformed first data strings, said one or more second data, and said one or more corrected third 
data strings are multiplexed by said multiplexing means; and 

outputting means for outputting said second coded signal through a second transmitting path at a target bit 
rate lower than said input bit rate of said first coded signal. 

The apparatus as set forth in claim 14, wherein said inputting means for inputting a first coded signal through a 
first transmitting path is operable to Input MPEG-2 transport streams, said demultiplexing means Is operable to 
demultiplex said MPEG-2 transport streams inputted by said inputting means into one or more transport stream 
packets having a coded video signal having a number of real Inputting bits as one or more first data strings, and 
said outputting means Is operable to output MPEG-2 transport streams having a video signal having a number of 
real outputting bits less than said number of real inputting bits of said coded video signal. 

The apparatus as set forth in claim 14, wherein said inputting means for inputting a first coded signal through a 
first transmitting path is operable to input coded multiplexed sound and moving picture sequence streams, said 
demultiplexing means is operable to demultiplex said coded multiplexed sound and moving picture sequence 
streams inputted by said Inputting means into one or more transport stream packets having a coded moving picture 
sequence signal having a number of real inputting bits as said one or more first data strings, and said outputting 
means is operable to output coded multiplexed sound and moving picture sequence streams having a coded 
moving picture sequence signal having a number of real outputting bits less than said number of real inputting bits 
of said coded moving picture sequence signal. 

The apparatus as set forth in claim 15 or claim 16, which further comprises reference time setting means for 
computing a value of system clock indicative of a start time of said demultiplexing means, on the basis of a first 
reference time information element contained in said first coded signal, and computing an initial value of said 
system clock for said second coded signal on the basis of said value of said system clock. 

The apparatus as set forth in any claim of claims 1 5 through 1 7, wherein said transfonning means further comprises: 

a video transport stream packet decoding unit for decoding said one or more transport stream packets having 
a coded video signal having a number of real inputting bits to reconstruct and output video PES packets; 
a video PES packet decoding unit for decoding said video PES packets decoded by said video transport stream 
packet decoding unit to reconstruct and output video elementary streams having a real inputting bits, decoding 
time management Information element DTS and presentation time management information element PTS of 
said video elementary streams, and PTS^DTS flags indicative of presence of said decoding time management 
information element DTS and said presentation time management information element PTS; 
a transforming unit for transforming said video elementary streams decoded and outputted by said video PES 
packet decoding unit into video elementary streams having a number of real outputting bits less than said 
number of real inputting bits of said video elementary streams; 

a video PES packet generating unit for generating transformed video PES packets on the basis of said trans- 
formed video elementary stream transformed by said transfonning unit, said decoding time management In- 
fonnation element DTS, said presentation time management Infomriation element PTS and said PTS.DTS 
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flags indicative of presence of said decoding time management Infomnation element DTS and said presentation 
time management Information element PTS decoded by said video PES packet decoding unit; and 
a video transport stream pacl<et generating unit for encoding said transformed video PES pacl<ets generated 
by said video PES paclcet generating unit to generate a video transfomned transport stream packets having a 
number of real outputting bits less than said n umber of real inputting bits of said video transport stream packets. 

19. The apparatus as set forth in any claim of claims 15 through 18, wherein said demultiplexing means is operative 
to demultiplex said first coded signal into transport stream packets having a coded audio signal as said one or 
more second data strings. 

20. The apparatus as set forth in any claim of claims 15 through 19, wherein said demultiplexing means is operative 
to demultiplex MPEG-2 transport streams Inputted at a predetemriined time interval by said inputting means into 
said one or more first data strings, said one or more second data strings, and said one or more third data strings, 
each having a number of real inputting bits, and said transfomiing means comprises: 

a video elementary stream decoding unit for decoding said one or more first data strings to reconstruct video 
elementary streams having a number of real inputting bits and other infomriation elementary streams, and 
separating said video elementary streams from said other information elementary streams; 
a video elementary stream transforming unit for transfomning said video elementary streams reconstructed 
and separated by said video elementary stream decoding unit to generate output video elementary streams 
having a number of real outputting bits less than said number of real inputting bits of said video elementary 
streams; and 

a transformed first data string generating unit for generating one or more transformed first data strings having 
a number of real outputting bits less than said number of real inputting bits of said one or more first data strings 
demultiplexed by said demultiplexing means on the basis of said output video elementary streams generated 
by said video elementary stream transfomning unit and said other information elementary steams reconstructed 
and separated by said video elementary stream decoding unit, whereby said multiplexing means is operable 
to multiplex said one or more transfomned first data strings generated by said transfomned first data string 
generating unit, said one or more second data strings demultiplexed by said demultiplexing means, and said 
one or more corrected third data strings corrected by said correcting means to generate a second coded signal 
to be outputted at said predetermined time interval. 

21 . The apparatus as set forth in claim 20, which further comprises a computing unit (A) being operative to: 

assume that a number of real outputting bits of said one or more second data strings contained in said second 
coded signal at said predetemnlned time interval is equal to a number of real Inputting bits of said one or more 
second data strings contained in said first coded signal at said predetemnined time interval; 
assume that a number of real inputting bits of said one or more third data strings contained in said second 
coded signal at said predetermined time interval is equal to said number of real outputting bits of said one or 
more corrected third data strings generated by said correcting means at said predetemnined time inten/al; 
subtract said number of real inputting bits of said one or more second data strings contained in said first coded 
signal at said predetemnined time interval and said number of real inputting bits of said one or more third data 
strings contained In said second coded signal at said predetemnined time Interval from a target number of 
outputting bits of all data strings contained in said second coded signal at said predetenmined time interval to 
generate a value A; 

subtract a total number of real outputting bits of one or more transformed first data strings generated by said 
transfomned first data string generating unit prior to said predetermined time interval from a total target number 
of outputting bits of one or more transfomned first data strings generated by said transformed first data string 
generating unit prior to said predetermined time interval to generate a value B; and 
compute a target number of outputting bits of said transformed first data strings generated at said predeter- 
mined time inten/al by said transformed first data string generating unit by adding said value A and said value B. 

22. The apparatus as set forth in claim 20 or claim 21 , which further comprises: 

computing unit (B) being operative to compute a target number of outputting bits of said output video elemen- 
tary streams on the basis of outputting bits of said second coded signal able to be outputted at said predeter- 
mined time Inten/al; 

computing unit (C) being operative to compute a reference ratio of outputting bits to inputting bits on the basis 
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of said target number of outputting bits of said output video elementary streams computed and said number 
of real inputting bits of said video elementary streams reconstructed by said video elementary stream decoding 
unit; and 

computing unit (D) being operative to compute a quantization scaling factor required for transforming said 
video elementary streams to generate said output video elementary streams on the basis of said reference 
ratio of outputting bits to inputting bits. 

23. The apparatus as set forth in claim 22, which further comprises: 

computing unit (E) being operative to compute a ratio of a total number of real outputting bits of said first one 
or more transformed data strings generated by said transfomned first data string generating unit prior to said 
predetemriined time interval to a total number of real outputting bits of said output video elementary streams 
prior to said predetermined time interval; 

computing unit (F) being operative to compute a target number of outputting bits of said output video elementary 
streams at said predetemriined time interval on the basis of said ratio computed by the computing unit (E); and 
computing unit (G) being operative to compute a reference ratio of outputting bits to inputting bits on the basis 
of said target number of outputting bits of said output video elementary streams and said number of real 
inputting bits of said video elementary streams decoded by said video elementary stream decoding unit. 

24. The apparatus as set forth in any claim of claims 20 through 23, wherein said transfomning means comprises: 

computing unit (H) being operative to compute a total number of real outputting bits of said output video ele- 
mentary streams generated by said video elementary stream transforming unit; 

computing unit (I) being operative to compute a sum of a target number of outputting bits of said output video 
elementary streams at said predetermined time interval and a total number of real outputting bits of said output 
video elementary streams which have been generated until the time when video elementary streams inputted 
prior to said predetermined time interval into a video ES buffer are consumed; 

judging unit (A) being operative to judge upon whether said total number of real outputting bits of said output 
video elementary streams computed by said computing unit (H) is greater than said sum computed by said 
computing unit (I); and 

control unit being operative to terminate said transforming means and starting said multiplexing means for 
processing said one or more transformed first data strings when it is judged that said total number of real 
outputting bits is greater than said sum by said judging unit (A). 

25. The apparatus as set forth in any claim of claims 15 through 24, wherein said multiplexing means comprises: 

computing unit (J) being operative to compute a PTS to PGR difference by subtracting a value of lastly past 
presentation time management information element PTS of said first coded signal from a value of a synchro- 
nous time information element PGR located in a head position of a data string of said one or more second 
data strings; 

computing unit (K) being operative to compute a passing time difference between passing time of said data 
string of said one or more second data strings in said first coded signal and passing time of said data string 
of said one or more second data strings in said second coded signal; 

judging unit (B) being operative to judge upon whether said PTS to PGR difference computed by said computing 
unit (J) is smaller than said passing time difference computed by said computing unit (K); and 
locating unit (A) being operative to locate said data string of said one or more second data strings in a rearward 
position of a data string of said one or more first data strings to be located in a rearward position of said data 
string of said one or more second data strings when it is judged tat said PTS to PGR difference computed by 
said computing unit (J) is smaller than said passing time difference computed by said computing unit (K) by 
said judging unit (B), and said outputting means is operative to output said second coded signal at said pre- 
determined time interval. 

26. The apparatus as set forth in any claim of claims 15 through 25, wherein said multiplexing means comprises: 

computing unit (L) being operative to compute a PGR to PTS difference by subtracting a value of a synchronous 
time information element PGR to be past subsequently in said first coded signal from a value of a presentation 
time management information element PTS of a just past data string of said one or more second data strings 
in said second coded signal; 
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computing unit (M) being operative to compute a passing time difference between passing time of said data 
string of said one or more second data strings in said first coded signal and passing time of said data string 
of said one or more second data strings in said second coded signal; 

judging unit (C) being operative to judge upon whethersaid PCRto PTS difference computed by said computing 
unit (L) is smaller ttian said passing time difference computed by said computing unit (M); and 
locating unit (B) being operative to locate said data string of said one or more second data strings in a forward 
position of a data string of said one or more first data strings to be located In a forward position of said data 
string of said one or more second data strings when it is judged that said PCR to PTS difference computed 
by said computing unit (L) is smaller than said passing time difference computed by said computing unit (M), 
and said outputting means is operative to output said second coded signal at said predetermined time interval. 

A computer program product comprising a computer usable storage medium having computer readable code em- 
bodied therein for transcoding a coded multiplexed sound and moving picture sequence, comprising: 

(a) computer readable program code for inputting a first coded signal through a first transmitting path at an 
input bit rate; 

(b) computer readable program code for demultiplexing said first coded signal inputted by said computer read- 
able program code (a) into one or more first data strings, one or more second data strings, and one or more 
third data strings, said one or more first data strings each having a number of real inputting bits; 

(c) computer readable program code for transfonning said one or more first data strings demultiplexed by said 
computer readable program code (b) into one or more first data strings having a number of real outputting bits 
less than said number of real inputting bits of said one or more first data strings, respectively; 

(d) computer readable program code for multiplexing said one or more transformed first data strings trans- 
fomfied by said computer readable program code (c). said one or more second data strings demultiplexed by 
said computer readable program code (b), and one or more corrected third data strings to generate a second 
coded signal; 

(e) computer readable program code for correcting said one or more third data strings demultiplexed by said 
computer readable program code (b), on the basis of said first coded signal, in accordance with a change of 
said second coded signal to generate said one or more corrected third data strings having a number of real 
inputting bits when said one or more transfomned first data strings, said one or more second data, and said 
one or more corrected third data strings are multiplexed by said computer readable program code (d); and 
(0 computer readable program code for outputting said second coded signal through a second transmitting 
path at a target bit rate lower than said input bit rate of said first coded signal. 

The computer program product as set forth in claim 27, wherein said computer readable program code (a) has 
computer readable program code (a2) for inputting MPEG-2 transport streams, said computer readable program 
code (b) has computer readable program code (b2) for demultiplexing said IVlPEG-2 transport streams inputted 
by said computer readable program code (a2) into one or more transport stream packets having a coded video 
signal having a number of real inputting bits as one or more first data strings, and said computer readable program 
code (f) has computer readable program code (f2) for outputting MPEG-2 transport streams having a video signal 
having a number of real outputting bits less than said number of real inputting bits of said coded video signal. 

The computer program product as set forth in claim 27, wherein said computer readable program code (a) has 
computer readable program code (a3) for inputting coded multiplexed sound and moving picture sequence 
streams, said computer readable program code (b) has computer readable program code (b3) for demultiplexing 
said coded multiplexed sound and moving picture sequence streams inputted by said computer readable program 
code (a3) into one or more transport stream pacl<ets having a coded moving picture sequence signal having a 
number of real inputting bits as said one or more first data strings, and said computer readable program code (f) 
has computer readable program code (f3) for outputting coded multiplexed sound and moving picture sequence 
streams having a coded moving picture sequence signal having a number of real outputting bits less than said 
number of real inputting bits of said coded moving picture sequence signal. 

The computer program product as set forth in claim 28 or claim 29, which further comprises: 

(g) computer readable program code for computing a value of system clock indicative of a start time of said 
computer readable program code (b), on the basis of a first reference time information element contained in 
said first coded signal; and 

(h) computer readable program code for computing an initial value of said system clock for said second coded 
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signal on the basis of said value of said system clock connputed by said computer readable program code (g). 

31 . The computer program product as set forth in any claim of claims 28 through 30, wherein said computer readable 
program code (c) further comprises: 

5 

(c51) computer readable program code for decoding said one or more transport stream packets having a 
coded video signal having a number of real inputting bits to reconstruct and output video PES packets; 
(c52) computer readable program code for decoding said video PES packets decoded by said computer read- 
able program code (c51) to reconstruct and output video elementary streams having a real inputting bits, 

^0 decoding time management infomnation element DTS and presentation time management Infomnatlon element 

PTS of said video elementary streams, and PTS^DTS flags indicative of presence of said decoding time man- 
agement infonnation element DTS and said presentation time management Information element PTS; 
(c53) computer readable program code for transforming said video elementary streams decoded and outputted 
by said computer readable program code (c52) into video elementary streams having a number of real out- 

*5 putting bits less than said number of real inputting bits of said video elementary streams; 

(c54) computer readable program code for generating transfonned video PES packets on the basis of said 
transformed video elementary stream transformed by said computer readable program code (c53), said de- 
coding time management Infomiation element DTS, said presentation time management infonnation element 
PTS and said PTS_DTS flags indicative of presence of said decoding time management information element 

20 DTS and said presentation time management infonnation element PTS decoded and outputted by said com- 

puter readable program code (c52); and 

(c56) computer readable program code for encoding said transfomried video PES packets generated by said 
computer readable program code (c54) to generate a transformed video transport stream packets having a 
number of real outputting bits less than said n umber of real inputting bits of said video transport stream packets. 

25 

32. The computer program product as set forth in any claim of claims 28 through 31 , wherein said computer readable 
program code (b) has computer readable program code for demultiplexing said first coded signal into transport 
stream packets having a coded audio signal as said one or more second data strings. 

30 33. The computer program product as set forth in any claim of claims 28 through 32, wherein said computer readable 
program code (b) has computer readable program code (b71) for demultiplexing MPEG-2 transport streams in- 
putted at a predetermined time interval by said computer readable program code (a) into said one or more first 
data strings, said one or more second data strings, and said one or more third data strings, each having a number 
of real inputting bits, and said computer readable program code (c) comprises: 

35 

(c71) computer readable program code for decoding said one or more first data strings to reconstruct video 
elementary streams having a number of real inputting bits and other infomiation elementary streams, and 
separating said video elementary streams from said other information elementary streams; 
(c72) computer readable program code for transforming said video elementary streams reconstructed and 
40 separated by said computer readable program code (c71 ) to generate output video elementary streams having 

a number of real outputting bits less than said number of real inputting bits of said video elementary streams; 
and 

(c73) computer readable program code for generating one or more transfomried first data strings having a 
number of real outputting bits less than said number of real inputting bits of said one or more first data strings 

45 demultiplexed by said computer readable program code (b71) on the basis of said output video elementary 

streams reconstructed by said computer readable program code (c72) and said other infonnation elementary 
steams reconstructed and separated by said computer readable program code (c71), whereby said computer 
readable program code (d) has computer readable program code (d71) for multiplexing said one or more 
transfomried first data strings generated by said computer readable program code (c73), said one or more 

50 second data strings demultiplexed by said computer readable program code (b71 ), and 

said one or more corrected third data strings con-ected by said computer readable program code (e) to generate 
a second coded signal to be outputted at said predetemnined time Interval. 

34. The computer program product as set forth in claim 33, wherein said computer readable program code (c) com- 
55 prises: 

(c81 ) computer readable program code for assuming that a number of real outputting bits of said one or more 
second data strings contained in said second coded signal at said predetemnined time interval is equal to a 
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number of real inputting bits of said one or more second data strings contained In said first coded signal at 
said predetermined time interval; 

(c82) computer readable program code for assuming that a number of real Inputting bits of said one or more 
third data strings contained In said second coded signal at said predetermined time interval is equal to said 
s number of real outputting bits of said one or more corrected third data strings generated by said computer 

readable program code (e) at said predetemnined time Interval; 

(c83) computer readable program code for subtracting said number of real Inputting bits of said one or more 
second data strings contained in said first coded signal at said predetemriined time interval and said number 
of real Inputting bits of said one or more third data strings contained in said second coded signal at said 
predetemiined time interval from a target number of outputting bits of all data strings contained in said second 
coded signal at said predetermined time interval to generate a value A; 

(c84) computer readable program code for subtracting a total number of real outputting bits of one or more 
transformed first data strings generated by said computer readable program code (c73) prior to said prede- 
temriined time interval from a total target number of outputting bits of one or more transformed first data strings 
generated by said computer readable program code (c73) prior to said predetermined time interval to generate 
a value B; and 

(c85) computer readable program code for computing a target number of outputting bits of said transformed 
first data strings generated at said predetemnlned time Interval by said computer readable program code (c73) 
by adding said value A and said value B. 

20 

35. The computer program product as set forth in claim 33 or claim 34, wherein said computer readable program code 
(c72) comprises: 

(c91) computer readable program code for computing a target number of outputting bits of said output video 
elementary streams on the basis of outputting bits of said second coded signal able to be outputted at said 
predetemiined time interval; 

(c92) computer readable program code for computing a reference ratio of outputting bits to inputting bits on 
the basis of said target number of outputting bits of said output video elementary streams computed by said 
computer readable program code (c91 ) and said number of real inputting bits of said video elementary streams 
reconstructed by said computer readable program code (c71 ); and 

(c93) computer readable program code for computing a quantization scaling factor required for transforming 
said video elementary streams to generate said output video elementary streams, on the basis of said refer- 
ence ratio of outputting bits to inputting bits computed by said computer readable program code (c92). 

35 36. The computer program product as set forth in claim 35, wherein said computer readable program code (c91) 
comprises: 

(c1 01 ) computer readable program code for computing a ratio of a total number of real outputting bits of said 
first one or more transfomied data strings generated by said computer readable program code (c73) prior to 
said predetermined time Interval to a total number of real outputting bits of said output video elementary 
streams prior to said predetermined time interval; and 

(c1 02) computer readable program code for computing a target number of outputting bits of said output video 
elementary streams at said predetemiined time interval on the basis of said ratio computed by said computer 
readable program code (c101), and said computer readable program code (c92) has computer readable pro- 
gram code (c1 03) for computing a reference ratio of outputting bits to inputting bits on the basis of said target 
number of outputting bits of said output video elementary streams computed by said computer readable pro- 
gram code (c102) and 

said number of real inputting bits of said video elementary streams reconstructed by said computer readable 
program code (c71). 

50 

37. The computer program product as set forth in any claim of claims 33 through 36, wherein said computer readable 
program code (c) comprises: 

(c1101) computer readable program code for computing a total number of real outputting bits of said output 
video elementary streams reconstructed by said computer readable program code (c72); 
(c1102) computer readable program code for computing a sum of a target number of outputting bits of said 
output video elementary streams at said predetemiined time Interval and a total number of real outputting bits 
of said output video elementary streams which have been generated until the time when video elementary 
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streams inputted prior to said predetermined time interval into a video ES buffer are consumed; 
(C1103) computer readable program code forjudging upon whether said total number of real outputting bits 
of said output video elementary streams computed by said computer readable program code (c1 1 01 ) Is greater 
than said sum computed by said computer readable program code (c11 02); and 

(c1 1 04) computer readable program code for terminating said computer readable program code (c) and starting 
said computer readable program code (d) for processing said one or more transformed first data strings when 
it is judged that said total number of real outputting bits is greater than said sum by said computer readable 
program code (c1103). 

38. The computer program product as set forth in any claim of claims 28 through 37, wherein said computer readable 
program code (d) comprises: 

(d1201) computer readable program code for computing a difference by subtracting a value of lastly past 
presentation time management infonnation element PTS of said first coded signal from a value of a synchro- 
nous time information element PGR located in a head position of a data string of said one or more second 
data strings; 

(d1 202) computer readable program code for computing a difference between passing time of said data string 
of said one or more second data strings in said first coded signal and passing time of said data string of said 
one or more second data strings in said second coded signal; 

(d1 203) computer readable program code for judging upon whether said difference computed by said computer 
readable program product (d1201) is smaller than said difference computed by said computer readable pro- 
gram code (d1202); and 

(d1 204) computer readable program code for locating said data string of said one or more second data strings 
in a reanward position of a data string of said one or more first data strings to be located in a reanward position 
of said data string of said one or more second data strings when it is judged that said difference computed by 
said computer readable program product {d1201) is smaller than said difference computed by said computer 
readable program code (d1202) by said computer readable program product (d1203), and said computer 
readable program code (f) has computer readable program code (f1201) for outputting said second coded 
signal at said predetermined time Interval. 

39. The computer program product as set forth in any claim of claims 28 through 38, wherein said computer readable 
program code (d) comprises: 

{d1 301 ) computer readable program code for computing a difference by subtracting a value of a synchronous 
time information element PCR to be past subsequently in said first coded signal from a value of a presentation 
time management Information element PTS of a just past data string of said one or more second data strings 
in said second coded signal; 

(d1 302) computer readable program code for computing a difference between passing time of said data string 
of said one or more second data strings in said first coded signal and passing time of said data string of said 
one or more second data strings in said second coded signal; 

(d 1 303) computer readable program code forjudging upon whether said difference computed by said computer 
readable program code (d1 301 ) is smaller than said difference computed by said computer readable program 
code (d1302); and 

(d1 304) computer readable program code for locating said data string of said one or more second data strings 
in a fonward position of a data string of said one or more first data strings to be located in a fonward position 
of said data string of said one or more second data strings when it is judged that said difference computed by 
said computer readable program code (d1301) is smaller than said difference computed by said computer 
readable program code (d1 302) by said computer readable program code (d1 303), and said computer readable 
program code (f) has computer readable program code (f1 301) for outputting said second coded signal at said 
predetemiined time interval. 
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